Organizing Research: brief subscriptions or research trips

5

1

Several of the big box providers like Ancestry, Find My Past, and British Origins offer limited-time subscriptions or limited free trials. I just took advantage of a month's cheap subscription at FindMyPast, and now I have a mountain of downloaded census and register images to sort out.

Usually I create a folder for these images that is named after the vendor, so I will know which online repository held those images. But if I'm not careful, the images can be saved to the wrong folder, or other things that I download in the same session can get saved to that folder by mistake.

What I would like to have is some kind of record of my download history. The ideal list would show the information that is in my Firefox download history -- the filename, the site it was downloaded from, and the date it was downloaded.

Searching Mozilla.org yields a lot of history cleaners, but I don't see anything that would allow me to export the download history to a file. Apparently the newer versions of Firefox only keep the history for a limited amount of time, so the window of opportunity to capture the data from there is small.

In the days of DOS, I knew how to pipe the contents of a directory to a file. If I had a list of the filenames, I could put them into a spreadsheet and then populate it with the information about where the files had come from, and eventually, with surname tags or other notes as a quick reminder of why I downloaded the file.

The reason I ask is that both Find My Past and The British Newspaper Archive keep logs of your records, but for the PPV records especially, the amount of time you can view them is limited. So I have a list of newspaper articles that I accessed last year, but since I didn't think to make notes on the BNA site, it isn't always evident what search terms made them appear, or why I collected them. So I spent a lot of time re-accessing some of the newspaper articles on Find My Past and downloading them again, trying to make notes as I went along.

My plan is to catalog and organize all the images and analyze them properly, but how do I preserve the ephemeral data about the repository in the meantime?

Jan Murphy

Posted 2014-06-09T19:14:02.190

Reputation: 22 994

Clarification: On Find My Past "My Records" shows a Record Summary which appears to be a the search parameters: "Nicholas, Cole (United Kingdom)", Cost, Last Viewed, First Viewed, Notes, Tags, and Certainty rating. Unless you make a note like "1891 Census" you can't see what you looked at without viewing the record again. The BNA shows the Newspaper title and date and the article title, but not the page or column. Ancestry.com shows downloaded and printed records via the Shoebox; historical records show name and collection, while newspapers and directories show title and publication date. – Jan Murphy – 2014-06-09T19:33:13.020

1Warning: there seems to be a limit beyond which no additional items are added to the "Downloaded and printed records" tab at ancestry. My shoebox stopped adding items more that a year ago; removing a few hundred items did nothing. A few months ago I cleared it out completely, more than 2500 items altogether, and still nothing new has been added. It was convenient but nothing I depended on, so I didn't find out more from ancestry. The "Saved records" tab works just fine, and is the only facet discussed in ancestry's FAQs for Shoebox. – bgwiehle – 2014-06-09T23:49:05.937

As far as I can tell from testing with a 'registered user' (free) account on Ancestry, if you view a record and put it into your Shoebox, you can see the detail page, but you can't view the corresponding record. However, if you create a tree and attach the record to it, you can come back to it after your sub/free access period is over and you can still view the image. – Jan Murphy – 2014-06-10T01:40:15.570

Would Zotero be overkill? – None – 2014-06-10T06:46:27.787

I tried Zotero a while ago, but gave it up when it became web-based. I also experimented a little bit with Outwit, but I haven't had much of a chance to learn it yet. – Jan Murphy – 2014-06-10T07:00:11.147

1@JanMurphy the standalone version still seems to be available. I'll confess I need to get to grips with it again.... so many things to do so little time. – None – 2014-06-10T08:45:22.327

Answers

3

Another way of doing this is to use OneNote or Evernote and grab the image of the page with the appropriate clipping tool and that stores the url that the image came from. These notes can be orgainsed into notebooks by site the information came from, surname or whatever.

I am not an expert user of OneNote only recently having started using it but it is a great way organise scraps of data that you gather for your study.

Both OneNote and Evernote are free applications.

Colin

Posted 2014-06-09T19:14:02.190

Reputation: 3 480

2

According to the mozillaZine article downloads.rdf

In Firefox (version 2.x and below) and Mozilla Suite/SeaMonkey, downloads.rdf is the (only) place that stores information (such as file name, URL, and date of download) about files downloaded. Its contents can be viewed and managed through the Download Manager and its associated settings. In Thunderbird, it is the (only) place that maintains a history of attachments opened or saved by double-clicking on files listed in the Attachments pane or by using the menu sequence "File -> Attachments -> [attachment name] -> Open". Thunderbird has no user interface for viewing or managing the contents.

Note: Starting in Firefox 3.0, download history is stored in downloads.sqlite instead of downloads.rdf.

The file downloads.rdf is readable with Firefox itself, and could be scraped to put the information into a spreadsheet or checklist.

The mozillaZine article on the profile folder says:

  • downloads.rdf Previous to 3.0 Download history. Can be deleted to resolve slow downloads or hangs. [24]
  • downloads.sqlite 3.0 and above; Previous to 26.0 Download history. [25] Removed in Firefox 26.0. [26]

An extension like SQLite Manager 0.8.1 would have allowed me to export the information in the file downloads.sqlite, but the current version of Firefox (29 -- I haven't yet upgraded to 30) doesn't use this file.

So where does Firefox 29/30 store this information? The Mozilla support forum:

Why did downloads.sqlite disappear from my profile when I upgraded to FF26?

The downloads.sqlite is no longer used in current Firefox releases.

Download history is stored as part of the normal browsing history (visited sites/links) in places.sqlite and a special annotation property identifies such an history item as a download item, so you can't separate them easily.

So it may not be possible to extract the information from the file places.sqlite. One possible solution posted on the MozillaZine forums last year is Export downloads history from Firefox SQLite DB.

But for me, creating a spreadsheet of filenames and copying the information by brute force (copy/paste) may be the most straightforward way to go.


Another strategy I've used is to capture pages with the Firefox extension Scrapbook, which keeps an editable screenshot of the page. If the service displays a message "you searched for [term 1 term 2]" then this saves your search terms along with the search results. The Scrapbook extension has a feature that will send you back to the live URL, so the actual URL for the captured page is retrievable.

Saving with Scrapbook is most useful for the hits from the British Newspaper Archives, but it doesn't solve the original problem of recording the source of the downloaded images and associating them with their captured transcriptions.

Jan Murphy

Posted 2014-06-09T19:14:02.190

Reputation: 22 994

Jan, that is the advantage of one of the note programmes below you can add the transcription and the image to the same notebook page rather than separate pages in Scrapbook. – Colin – 2014-06-10T10:05:53.047

1

Re: lists of (local) filenames - I use Karen's Directory Printer to generate reports (as txt files), which can be copied to Excel. This versatile program is still available at http://www.karenware.com/powertools/ptdirprn.asp, although some parts of the website don't work right anymore (the author died in Apr 2011).

– bgwiehle – 2014-06-11T00:48:40.117