The highlight of my digital archives week was the discovery of the Digital Curation Exchange. I promptly joined and began to explore their ‘space for all things ‘digital curation’ ‘. This led me to a fabulous list of resources, including a set of syllabi for courses related to digital curation. Each link brought me to an extensive reading list, some with full slide decks related to weekly in classroom presentations. My ‘to read’ list has gotten much longer – but in a good way!

On other days recently I have found myself involved in all of the following:

• review of metadata standards for digital objects
• creation of internal guidelines and requirements documents
• networking with those at other institutions to help coordinate site visits of other digitization projects
• records management planning and reviews
• learning about the OCR software available to our organization
• contemplation of the web archiving efforts of organizations and governments around the world
• reviewing my organization’s social media policies
• listening to the audio of online training available from PLANETS (Preservation and Long-term Access through NETworked Services)
• contemplation of the new Journal of Digital Media Management and their recent call for articles

My new favorite quote related to digital preservation comes from What we reckon about keeping digital archives: High level principles guiding State Records’ approach from the State Records folks in New South Wales Australia, which reads:

We will keep the Robert De Niro principle in mind when adopting any software or hardware solutions: “You want to be makin moves on the street, have no attachments, allow nothing to be in your life that you cannot walk out on in 30 seconds flat if you spot the heat around the corner” (Heat, 1995)

In other words, our digital archives technology will be designed to be sustainable given our limited resources so it will be flexible and scalable to allow us to utilise the most appropriate tools at a given time to carry out actions such as creation of preservation or access copies or monitoring of repository contents, but replace these tools with new ones easily and with minimal cost and with minimal impact.

I like that this speaks to the fact that no plan can perfectly accommodate the changes in technology coming down the line. Being nimble and assuming that change will be the only constant are key to ensuring access to our digital assets in the future.

This post is from from: Spellbound Blog.

Day of Digital Archives

I will break this down into a number of phases:

• Phase 1: Hardware
• Phase 2: Pull the data off the disk
• Phase 3: Extract the files from the disk image
• Phase 4: Migrate or Emulate

Phase 1: Hardware

Before you do anything else, you actually need a 5.25″ floppy drive of some kind connected to your computer.  I was lucky – a friend had a floppy drive for us to work with. If you aren’t that lucky, you can generally find them on eBay for around $25 (sometimes less). A friend had been helping me by trying to connect the drive to my existing PC – but we could never get the communications working properly. Finally I found Device Side Data’s 5.25″ Floppy Drive Controller which they sell online for $55. What you are purchasing will connect your 5.25 Floppy Drive to a USB 2.0 or USB 1.1 port. It comes with drivers for connection to Windows, Mac and Linux systems.

If you don’t want to mess around with installing the disk drive into our computer, you can also purchase an external drive enclosure and a tabletop power supply. Remember, you still need the USB controller too.

Update: I just found a fantastic step-by-step guide to the hardware installation of Device Side’s drive controller from the Maryland Institute for Technology in the Humanities (MITH), including tons of photographs, which should help you get the hardware install portion done right.

Phase 2: Pull the data off the disk

The next step, once you have everything installed, is to extract the bits (all those ones and zeroes) off those floppies. I found that creating a new folder for each disk I was extracting made things easier. In each folder I store the disk image, a copy of the extracted original files and a folder named ‘converted’ in which to store migrated versions of the files.

Device Side provides software they call ‘Disk Image and Browse’. You can see an assortment of screenshots of this software on their website, but this is what I see after putting a floppy in my drive and launching USB Floppy -> Disk Image and Browse:

You will need to select the ‘Disk Type’ and indicate the destination in which to create your disk image. Make sure you create the destination directory before you click on the ‘Capture Disk File Image’ button. This is what it may look like in progress:

Fair warning that this won’t always work. At least the developers of the software that comes with Device Side Data’s controller had a sense of humor. This is what I saw when one of my disk reads didn’t work 100%:

If you are pressed for time and have many disks to work your way through, you can stop here and repeat this step for all the disks you have on hand.

Phase 3: Extract the files from the disk image

Now that you have a disk image of your floppy, how do you interact with it? For this step I used a free tool called Virtual Floppy Drive. After I got this installed properly, when my disk image appeared, it was tied to this program. Double clicking on the Floppy Image icon opens the floppy in a view like the one shown below:

It looks like any other removable disk drive. Now you can copy any or all of the files to anywhere you like.

Phase 4: Migrate or Emulate

The last step is finding a way to open your files. Your choice for this phase will depend on the file formats of the files you have rescued. My files were almost all WordStar word processing documents. I found a list of tools for converting WordStar files to other formats.

The best one I found was HABit version 3.

It converts Wordstar files into text or html and even keeps the spacing reasonably well if you choose that option. If you are interested in the content more than the layout, then not retaining spacing will be the better choice because it will not put artificial spaces in the middle of sentences to preserve indentation. In a perfect world I think I would capture it both with layout and without.

Summary

So my rhythm of working with the floppies after I had all the hardware and software installed was as follows:

• create a new folder for each disk, with an empty ‘converted’ folder within it
• insert floppy into the drive
• run DeviceSide’s Disk Image and Browse software (found on my PC running Windows under Start -> Programs -> USB Flopy)
• paste the full path of the destination folder
• name the disk image
• click ‘Capture Disk Image’
• double click on the disk image and view the files via vfd (virtual floppy drive)
• copy all files into the folder for that disk
• convert files to a stable format (I was going from WordStar to ASCII text) and save the files in the ‘converted’ folder

These are the detailed instructions I tried to find when I started my own data rescue project. I hope this helps you rescue files currently trapped on 5 1/4″ floppies. Please let me know if you have any questions about what I have posted here.

Update: Another great source of information is Archive Team’s wiki page on Rescuing Floppy Disks.

This post is from from: Spellbound Blog.

Rescuing 5.25″ Floppy Disks from Oblivion

Presenters:

• Adele McAlear author of Death and Digital Legacy
• Dazza Greenwood author of Civics.com
• Evan Carroll and John Romano, co-authors of Your Digital Afterlife
• Jesse Davis cofounder of Entrustet

There was a lot to take in here. You can listen to the full audio of the session or watch a recording of the session’s live stream (the first few minutes of the stream lacks audio).

A quick and easy place to start is this lovely little video created as part of the promotion of Your Digital Afterlife – it gives a nice quick overview of the topic:

Also take a look at the Visual Map that was drawn by Ryan Robinson during the session – it is amazing! Rather than attempt to recap the entire session, I am going to just highlight the bits that most caught my attention:

Laws, Policies and Planning
Currently individuals are left reading the fine print and hunting for service specific policies regarding access to digital content after the death of the original account holder. Oklahoma recently passed a law that permits estate executors to access the online accounts of the recently deceased – the first and only state in the US to have such a law. It was pointed out during the session that in all other states, leaving your passwords to your loved ones is you asking them to impersonate you after your death.

Facebook has an online form to report a deceased person’s account – but little indication of what this action will do to the account. Google’s policy for accessing a deceased person’s email requires six steps, including mailing paper documents to Mountain View, CA.

There is a working group forming to create model terms of service – you can add your name to the list of those interested in joining at the bottom of this page.

What Does Ownership Mean?
What is the status of an individual email or digital photo? Is it private property? I don’t recall who mentioned it – but I love the notion of a tribe or family unit owning digital content. It makes sense to me that the digital model parallel the real world. When my family buys a new music CD, our family owns it – not the individual who happened to go to the store that day. It makes sense that an MP3 purchased by any member of my family would belong to our family. I want to be able to buy a Kindle for my family and know that my son can inherit my collection of e-books the same way he can inherit the books on my bookcase.

Remembering Those Who Have Passed
How does the web change the way we mourn and memorialize people? Many have now had the experience of learning of the passing of a loved one online – the process of sorting through loss in the virtual town square of Facebook. How does our identity transform after we are gone? Who is entitled to tag us in a photo?

My family suffered a tragic loss in 2009 and my reaction was to create a website dedicated to preserving memories of my cousin. At the Casey Feldman Memories site, her friends and family can contribute memories about her. As the site evolved, we also added a section to preserve her writing (she was a journalism student) – I kept imagining the day when we realized that we could no longer access her published articles online. I built the site using Omeka and I know that we have control over all the stories and photos and articles stored within the database.

It will be interesting to watch as services such as Chronicle of Life spring up claiming to help you “Save your memories FOREVER!”. They carefully explain why they are a trustworthy digital repository and why they backup their claims with a money-back guarantee.

For as little as $10, you can preserve your life story or daily journal forever: It allows you to store 1,000 pages of text, enough for your complete autobiography. For the same amount, you could also preserve less text, but up to 10 of your most important photos. – Chronicle of Life Pricing

Privacy
There are also some interesting questions about privacy and the rights of those who have passed to keep their secrets. Facebook currently deletes some parts of a profile when it converts it to a ‘memorial’ profile. They state that this is for the privacy of the original account holder. If users are ultimately given more power over the disposition of their social web presence – should these same choices be respected by archivists? Or would these choices need to be respected the way any other private information is guarded until some distant time after which it would then be made available?

Conculsion
Thanks again to all the presenters – this really was one of the best sessions for me at SXSWi! I loved that it got a whole different community of people thinking about digital preservation from a personal point of view. You may also want to read about Digital Death Day – one coming up in May 2011 in the San Francisco Bay Area and another in September 2011 in the Netherlands.

Image credit: Excerpt from Ryan Robinson’s Visual Map created live during the SXSW session.

This post is from from: Spellbound Blog.

SXSWi: You’re Dead, Your Data Isn’t: What Happens Now?

There is a dedicated Census History area on the Census website, as well as a section of the 2010 website called The Big Count Archive. While I like the wide range of 2010 Census Posters – the 1940 census poster shown here (thank you Library of Congress) is just so striking.

I also loved the videos I found, especially when I realized that they were all available on YouTube – uploaded by a user named JasonGCensus. I am not clear on the relationship between JasonGCensus and the official U.S. Census Bureau’s Channel (which seems focused on 2010 Census content), but there are some real gems posted there.

For example, in the 1970 Census PSA shown below we learn about the privacy of our census data: “Our separate identities will be lost in the process which is concerned only with what we say, not who said it”. We are shown technology details – complete with old school beeping and blooping computer sounds. (NOTE: this video is also available on Census.gov, but I saw no way to embed that video here – hence my cheer at finding the same video on YouTube)

For the 1960 census, a PSA explains the new FOSDIC technology which removed the need for punch-cards. With the tagline ‘Operation Rollcall, USA’, the ad presents our part in “this enterprise” as cooperation with the enumerators. In the 1980 PSA the tag line is ‘Answer the Census: We’re counting on you!’ and stresses that it is kept confidential and is used to provide services to communities. By the time you get to the 1990 and 2000 PSAs we see more stress on the benefits to communities that fill out the census and less stress on how the census is actually recorded.

I also found some lovely census images in the Library of Congress Prints and Photographs catalog including the image shown here and:

• an 1870 Wood Engraving
• an 1890 Cartoon
• a 1910 Postcard

Exploring the area of Census.gov dedicated to the 2010 census made me wonder what was available online for the 2000 census.

Wayback Machine to the rescue! They have what appears to be a fairly deep crawl of the 2000 Census.gov site dating from March of 2000. For example – the posters section seems to include all the images and PDFs of the originals. I even found functional Quicktime videos in the Video Zone, like this one: How America Knows What America Needs.

The ten year interval makes for a nice way to get a sense of the country from the PR perspective. What did the Census Bureau think was the right way to appeal to the American public? Were we more intrigued by the latest technology or worried about our privacy? Did they need to communicate what the census is used for? Or was it okay to simply express it as an American’s duty? I appreciate the ease with which I can find and share the resources above. Great fun.

And for those of you in the United States, please consider this my personal encouragement to fill out your census forms!

Update: The WashingtonPost has an interesting article about the ‘Snapshot of America’ series of promotional videos for the 2010 census. Definitely an interesting contrast to the videos I reviewed for this post.

This post is from from: Spellbound Blog.

Encouraging Participation in the Census

The Official Google Reader Blog recently announced a new feature that will let users watch any page for updates. The way this works is that you add individual URLs to your Google Reader account. Just as with regular RSS feeds, when an update is detected – a new entry is added to that subscription.

My thinking is that this could be a really useful tool for archivists charged with preserving websites that change gradually over time, especially those fairly static sites that change infrequently with little or no notice of upcoming changes. If a web page was archived and then added to a dedicated Google Reader account, the archivist could scan their list of watch pages daily or weekly. Changes could then trigger the creation of a fresh snapshot of the site.

I will admit that there have been services out there for a while that do something similar to what Google has just rolled out. I personally have used Dapper.net to take a standard web page and generate an RSS feed based on updates to the page (sound familiar?). One Dapper.net feed that I created and follow is for the news archive page for the International Red Cross and can be found here. What is funny is that now they actually have an official RSS feed for their news that includes exactly what my Dapper.net feed harvested off their news archive page – but when I built that Dapper feed there was no other way for me to watch for those news updates.

There are lots of different tools out there that aim to archive websites. Archive-It is a subscription based service run by Internet Archive that targets institutions and will archive sites on demand or on a regular schedule. Internet Archive also has an open source crawler called Heritrix for those who are comfortable dealing with the code. Other institutions are building their own software to tackle this too. Harvard University has their own Web Archive Collection Service (WAX). The LiWA (Living Web Archives) Project is based in Germany and aims to “extend the current state of the art and develop the next generation of Web content capture, preservation, analysis, and enrichment services to improve fidelity, coherence, and interpretability of web archives.” One could even use something as simple as PDFmyURL.com – an online service that turns any URL into a PDF (be sure to play with the advanced options to make sure you get a wide enough snapshot). I know there are many more possibilities – these just scratch the surface.

What I like about my idea is that it isn’t meant to replace these services but rather work in tandem with them. The Internet Archive does an amazing job crawling and archiving many web pages – but they can’t archive everything and their crawl frequency may not match up with real world updates to a website. This approach certainly wouldn’t scale well for huge websites for which you would need to watch for changes on many pages. I am picturing this technique as being useful for small organizations or individuals who just need to make sure that a county government website makeover or a community organization’s website update doesn’t get lost in the shuffle. I like the idea of finding clever ways to leverage free services and tools to support those who want to protect a particular niche of websites from being lost.

Image Credit: The RSS themed image above is by Matt Forsythe.

This post is from from: Spellbound Blog.

Leveraging Google Reader’s Page Change Tracking for Web Page Preservation

• Weather balloon observations received from Lilongwe, Malawi (Africa) from 1968-1991: all the red on these charts represents data rescued by IEDRO — an increase from only 30% of the data available to over 90%.
• Data rescue statistics from around the world

They do this work for many reasons – to improve understanding of weather patterns to prevent starvation and the spread of disease, to ensure that structures are built to properly withstand likely extremes of weather in the future and to help understand climate change. Since the theme for the day is climate change, I thought I would include a few excerpts from their detailed page on climate change:

“IEDRO’s mandate is to gather as much historic environmental data as possible and provide for its digitization so that researchers, educators and operational professionals can use those data to study climate change and global warming. We believe, as do most scientists, that the greater the amount of data available for study, the greater the accuracy of the final result.

If we do not fully understand the causes of climate change through a lack of detailed historic data evaluation, there is no opportunity for us to understand how humankind can either assist our environment to return to “normal” or at least mitigate its effects. Data is needed from every part of the globe to determine the extent of climate change on regional and local levels as well as globally. Without these data, we continue to guess at its causes in the dark and hope that adverse climate change will simply not happen.”

So, what does this data rescue look like? Take a quick tour through their process – from organizing papers, photographing each page, the transcription of all data and finally upload of this data to NOAA’s central database. These data rescue efforts span the globe and take the dedicated effort of many volunteers along the way. If you would like to volunteer to help, take a look at the IEDRO listings on VolunteerMatch.

This post is from from: Spellbound Blog.

Blog Action Day 2009: IEDRO and Climate Change

Digital lives is a joint project of UCL, British Library and University of Bristol

What? We need a better understanding of how people manage digital collections on their laptops, pdas and home computers. This is important due to the transition from paper-based personal collections to digital collections. The hope is to help people manage their digital archives before the content gets to the archives.

How? Talk to people with in-depth narrative interview. Ask people of their very first memories of information technology. When did they first use the computer? Do they have anything from that computer? How did they move the content from that computer? People enjoyed giving this narrative digital history of their lives.

Who? 25 interviewees – both established and emerging people whose works would or might be of interest to repositories of the future.

Findings?

• They created a detailed flowchart of users’ reported process of document manipulation.
• Common patterns in use of email showed that people used email across all these platforms and environments. Preserving email is not just a case of saving one account’s messages:
  • work email
  • Gmail/Yahoo
  • mails via Facebook
  • Twitter
• Documented personal information styles that relate skills dimension to data security dimension.

The one question I caught was from someone who asked if they thought people would stop using folders to organize emails and digital files with the advent of easy search across documents. The speaker answered by mentioning the revelations in the paper Don’t Take My Folders Away!. People like folders.

My Thoughts

This session got me to think again about the SAA2008 session that discussed the challenges that various archivists are facing with hybrid literary collections. Matthew Kirschenbaum also pointed me to MITH’s white paper: Approaches to Managing and Collecting Born-Digital Literary Materials for Scholarly Use.

I am very interested to see how ideas about preserving personal digital records evolve. For example, what happens to the idea of a ‘draft’ in a world that auto-saves and versions documents every few minutes such as Google Documents does?

With born digital photos we run into all sorts of issues. Photos that are simultaneously kept on cameras, hard drives, web based repositories (flickr, smugmug, etc) and off-site backup (like mozy.com). Images are deleted and edited differently across environments as well. A while back I wrote a post considering the impact of digital photography on the idea of photographic negatives as the ‘photographers’ sketchbooks’: Capa’s Found Images and Thoughts on Digital Photographers’ Sketchbooks.

I really liked the approach of this project in that it looked at general patterns of behavior rather than attempting to extrapolate from experiences of archivists with individual collections. This sort of research takes a lot of energy, but I am hopeful that basically creating these general user profiles will lead to best practices for preserving personal digital collections that can be applied easily as needed.

As is the case with all my session summaries from DH2009, please accept my apologies in advance for any cases in which I misquote, overly simplify or miss points altogether in the post above. These sessions move fast and my main goal is to capture the core of the ideas presented and exchanged. Feel free to contact me about corrections to my summary either via comments on this post or via my contact form.

This post is from from: Spellbound Blog.

DH2009: Digital Lives and Personal Digital Archives

I really want a t-shirt with the Bit-Rot guy on it!

This post is from from: Spellbound Blog.

Another Thrilling Digital Adventure With Team Digital Preservation

One of the films available from the archives is the 1975 action feature Doc Savage: The Man of Bronze. Embedded below is a 30 second clip showing Doc Savage entering his “Fortress of Solitude”. They could have made it easier for me to embed this (I had to go figure out how to embed FLV files into this blog post) – but I am happy that they let me embed it at all. If you don’t see a video below, you probably need to install adobe’s shockwave. You can always go watch the clip on the Doc Savage page (click on Video Trailers & Clips).

Each film page carefully notes “This film has been manufactured from the best-quality video master currently available and has not been remastered or restored specifically for this DVD and On Demand release.” and then directs the customer to view the preview clip to evaluate the film’s quality.

The details comes out when we dig into the Warner Archive FAQ. It is here that we learn that the DVDs we can purchase for $19.95 are produced “on-demand”. How are they different from the DVD’s you buy at the store?

DVD’s produced on-demand are similar to, but not quite same as, DVD’s you’d buy at the local video store. DVD movies you buy at the local video outlet are manufactured from a mold via a stamping process whereas on-demand DVDs are “burned”. Each carries information read by the DVD player, but the physical properties of the two are different.

Most DVD players are compatible with both commercial DVD-Video and one or more of the “recordable DVD formats. Our on-demand DVD’s are manufactured using the most widely accepted format, DVD-R.

They also answer this question about copying the DVDs:

Q: I’m trying to make a few extra copies of my DVD, for “safe keeping” and for a surprise present to my mom. When I copied the disc it was un-playable. Why is that? And what can I do about it?

A: This DVD on-demand disc was recorded using CSS encryption. CSS is designed to prevent unauthorized reproduction of the DVD. We’re delighted that you’d like to surprise your mother with the gift of a Warner Bros classic movie. May we suggest she’d like an officially produced and packaged DVD even more? As such we welcome your visit back to the Warner.com classic store at any time.

In addition to being able to purchase DVD-Rs with CSS encryption, many of the archives films permit a download option. Archives movie downloads appear to cost $14.95. The Digital Products FAQ explains the details, but these are the highlights of what comes along with that $5 in savings:

• Downloads are protected by DRM
• Downloads only play on MS Windows boxes – no Mac or Linus support
• You can burn the movie to a CD or DVD, but they “are Digital Rights Management (DRM) protected, so you will only be able to watch the video on the computer or device on which it was originally purchased.”

I give a big thumbs up to Warner Brothers for coming up with a way to leverage their archives. I am less impressed with the non-open format and DRM restrictions they are placing on both the DVD-Rs and downloads. A model that states that a purchased download can be played as often as I want – but requires a specific operating system and only permits play on the same machine from which I made the purchase seems untenable. If I were to buy one of these films, I would spend the extra $5 and get the DVD-R which at least can be played on multiple machines, even if it can never be copied!

This post is from from: Spellbound Blog.

Warner Brothers Archive DVDs: Classic Movies On-Demand

My first find has me giving a big hats off to IBM. They have a page dedicated to IBM Women in Technology, but the real fun is in digging through the persona pages listed in the IBM Women in Technology International (WITI) hall of fame. You can watch oral history interviews with women like Frances Allen,  an “expert in the field of optimizing compilers”, or Caroline Kovac, who “oversees the development of cutting-edge information technology at IBM for the life sciences market”.

Beyond IBM’s offerings I ran into a classic challenge – how do you find archival collections specifically about women in technology? A visit to the American Institute of Physic’s archive found me a photo mini-exhibits of of Marie Curie and Maria Goeppert Mayer. A search for “woman scientists” on the Online Archive of California (OAC) found these:

• Contributions of 20th Century Women to Physics : Records of the UCLA Website 1912-2001: The records include documentation of the original papers in which discoveries were first reported, biographical material, including some photographs, and descriptions vetted by Field Editors.
• Katherine Esau papers: The Katherine Esau papers represent the entire body of plant anatomy research Esau conducted from 1924 when she began research on curly top virus in sugar beets for the Spreckels Sugar Company to 1991 when she published her last article. The collection includes correspondence, research notes, photographs, biographical material, objects, and printed matter.

The challenge in finding collections like these is that you need to hunt through each institutions collections. Looking for the records of a specific individual is easiest, but finding collections in general relating to women and technology is a lot harder. The first collection listed above from OAC has the subject “Women in physics –Archival resources” assigned to it, which seems very useful until you realize that it is the only collection assigned this subject in all of OAC.

I want to leave you with the thought that preserving the notes and writing of young innovative women who are passionate about technology is what will let future generations read their words just as young women can read and be inspired by the words of Ada Lovelace today.

Want to read some of Ada’s writing? Get your hands on a copy of Ada, the Enchantress of Numbers: A Selection from the Letters of Lord Byron’s Daughter and Her Description of the First Computer. Want to read something a bit more contemporary that is halfway between memoir and eclectic visit to the depths of software programming, then try Ellen Ullman’s  Close to the Machine: Technophilia and Its Discontents.

Technorati Tag: ALD09post

This post is from from: Spellbound Blog.

Archiving Women in Technology: A Tribute to Ada Lovelace