Database design

[xilman]

Anyone here, other than fivemack and I, have a significant number of astronomical images? Despite not having even a primitive webcam attached to even the most rudimentary telescope, I've already accumulated around a thousand images, roughly 10% from Tom. A few hundred more, >600 of them being 0.5ms images of Sirius, were provided by Paul Fellows who is also a member of the Cambridge Astronomical Association. The results of our collaboration can be found in the latest CAA newsletter.

I noticed that Paul had a few difficulties finding historical images in his file systems and before that I'd already recognized the likelihood that I would too in the future. The standard image format, FITS, supports rich meta-data about the image(s) a file contains but it is not easily searchable across many images. Much the same can be said about the NEF format images which Tom provided.

Accordingly I've been thinking about a SQL database and have written a preliminary PostgreSQL schema. The schema is doubtless incomplete, too baroque and/or just plain wrong but the aim is to be able to locate any images which were taken from a particular place, in a particular time period, contain a particular celestial co-ordinate, (type of) celestial object, constellation, photometry band and so on. Explicitly not required is to duplicate all the image meta-data nor all the data contained in the likes of the NOMAD-1 catalogue.

If anyone else is interested in such a database and would like to get involved, please say so in this thread.

[Uncwilly]

Originally Posted by xilman

Anyone here, other than fivemack and I, have a significant number of astronomical images?
....
The standard image format, FITS, supports rich meta-data about the image(s) a file contains but it is not easily searchable across many images. Much the same can be said about the NEF format images which Tom provided.
.....
Accordingly I've been thinking about a SQL database and have written a preliminary PostgreSQL schema. The schema is doubtless incomplete, too baroque and/or just plain wrong but the aim is to be able to locate any images which were taken from a particular place, in a particular time period, contain a particular celestial co-ordinate, (type of) celestial object, constellation, photometry band and so on. Explicitly not required is to duplicate all the image meta-data nor all the data contained in the likes of the NOMAD-1 catalogue.

If anyone else is interested in such a database and would like to get involved, please say so in this thread.

I don't know if any standard image catalogue software would be useful. I have threatened myself with installing some years ago (I downloaded a huge number of images of trains and autos from usenet).

I will be in the same room this Friday with NASA staff from JPL. Their Planetary Data System might be a cannon for your fly. I can see if one or more would be willing to talk to you. Last year at the event I was attending there were 2 JPL projects being presented (along with others): one wrt data set management, large data set transferring (data sets in the 10's of GB size [or even the 100's] that needed to be made available), & hosting; and an automated Mars crater identification and data extraction system (taking image sets and determining size, depth, etc. IIRC correctly this also worked with input from the Zooniverse's crater mappers.) I talked with one of the JPL staff a few months ago (very informally).