software | Spellbound Blog

SEO Evaluation of an Archival Website: Looking at UMBC’s Digital Collections

September 12, 2009 8 Comments

Each week brings announcements of archives launching new websites. Today both my email and Twitter told me about University of Maryland, Baltimore County’s new Digital Collections site. Who can resist peeking at new materials available online?

I have spent much of the past year learning the details of Search Engine Optimization. Usually shortened to SEO, this simply refers to the use of techniques which improve the traffic sent to a website via organic search. Want your webpage to show up at the top of the list for a specific search in Google? You want to work on your SEO.

So when I look at new archives website, I can’t help but keep an eye open for how well the site is optimized for search engines.

I hope that UMBC will forgive me for nitpicking their new site. A lot of their choices are great for SEO, but they also have room for improvement.

Things Done Well for SEO

Home Page Title & Description: The site’s home page has a good meta description. This is the text displayed below the link on a search results page – as shown below:
Unique Page Titles At Collection Level: Each photography collection homepage has a unique page title and a nice block of explanatory text. Google can only read words – so the more unique text on a page, the better the job Google can do in figuring out what your page is about. Example: Ardsley Park Album
Good anchor text: (also known as link text) The words used in anchor text tells search engines information about the destination page. For example, the blue text below is anchor text.

Areas for SEO Improvement

Unique Page Titles At Item Level: Individual images and documents all use a generic page title such as ‘UMBC | Digital Archive | Document Viewer’. Document Example: Accidental Death of an Anarchist Image Example: 10 year old Bootblack
H1 Tags: In the HTML of each page, the dominant heading of the page should use the <h1> tag. This helps Google know the phrase you are targeting with this page. It is your 2nd best place to emphasize your content after the page title. In the case of the item pages, there seems to often be a headline type title at the top of the page – but it currently is not an demarcated with an <h1> tag.
Think About Search Results and Indexing: Pages displaying results of internal searches on your site are not likely to be useful as indexed pages in Google. The thinking here is that they can dilute the focus on the item and collection level pages on your site if Google also has many search results pages in the index. If UMBC wanted their search pages to be indexed, then those pages’ URLs should be simplified and the search results pages need a page title that somehow includes the search criteria. There are two ways that I know of to disable this indexing – blocking via the site’s robots.txt file or via a robots meta tag in the header of the search results page. Both of these methods tell obliging search engines to not crawl certain parts of your site.

Final Thoughts

There are plenty of other things that UMBC could do to support this new website. They could create an XML sitemap of all their pages and submit it to Google (maybe they already have). They might re-title some of their pages based on using a tool like Google Insight to see what variations of a phrase is searched on most frequently. My goal here was to give you a taste of the sorts of things that catch my eye. Also, SEO is still more of an art than a science – so you will sometimes notice that what one SEO expert recommends is the opposite of what the next expert would tell you.

In many cases changes, such as the Unique Page Title at the Item Level mentioned above, may not even be possible due to software or programmer resource limitations. The trick is to take advantage of every option that is available. There are also trade-offs to be made. UMBC’s site provides some very slick interfaces for viewing the details of a group of documents, such as theater programs and other materials related to a theatrical production. The imlementation elegantly handles the situation of multiple scanned images which relate to a coherent set of documents. Sometimes you can’t have both your innovative UI and perfect SEO. Then it gets down to what your goals are for your website. Are you trying to make a specific community of existing users happy by providing them with tools they can use? Or does your mission focus more on reaching out to a broader audience?

There is no silver bullet to search engine optimization. It just takes knowledge of the available tools and techniques combined with a willingness to keep learning and experimenting. Like the ‘Do-It-Yourself-Woman‘ pictured above in the Nationaal Archief‘s photo I found out on the Flickr Commons, you too can learn the basics and do-it-yourself. A great starting point is Google’s free SEO Guide. Also, please remember that the best time to plan your SEO strategy is before you have built your site in the first place!

I would love to do research on how much progress archives websites can make in their organic search traffic after SEO improvements. My thinking is to take a snapshot of a month of analytics (the statistics that tell you how many people are visiting your website) and then apply some SEO inspired changes. After a suitable delay (it takes some time for SEO to do its job) we consider another month of analytics to determine any change in organic traffic.

Do you want me to do a quick review of your archives website to see if there is room for SEO improvement? Please contact me or add a comment to this post. I feel like there is a conference presentation in all this if we can find a good set of websites to optimize.

Finally, thank you to unsuspecting UMBC – your new website really is beautiful.

Image credit: Doe-het-zelf vrouw /Do-it-yourself-woman from Nationaal Archief on Flickr Commons.

A History of Our Own, Representing Communities and Identities on the Web (SAA09: Session 202)

September 8, 2009 1 Comment

Andrew Flinn, University College London (UCL), was the second speaker during SAA09’s Session 202 with his presentation ‘A History of Our Own, Representing Communities and Identities on the Web’. Flinn began with the idea that archives are “a place for creating and re-working memory”. While independent community archives are constituted around many purposes, Flinn’s main interest is in communities focused on absences and mis-representation of a group or event in history. Communities in which there is a cultural, politcal, or artistic activism. Some of these communities may be considered ‘movements’.

How should/can archivists support local archiving activities?

Part of the challenge of online communities is the need to capture the interactions in order to not loose the full picture. The National Listing of Community Archives in the UK‘s website states that they “seek to document the history of all manner of local, occupations, ethnic, faith and other diverse communities”.

The UCL’s International Centre for Archives and Records Management Research and User Studies (ICARUS) “brings together researchers in user access and description, community archives and identity, concepts and contexts of records and archives, and information policy”. Flinn is the Principal Investigator on the ICARUS project Community archives and identities which focuses on in depth interviews of 4 institutions which are “documenting and sustaining community heritage”.

These are some example online community sites:

rukus – black gblt archives
Moroccan Memories in Britain
eside community – east side working class community in London

Main Findings

proceed from a position that ‘knowing your own history’ is beneficial their communities as well as to the public at large
the quality of the work is done by individual passion and sacrifice, voluntary
there is ambivalence to/about the mainstream archives sector — keen to work with mainstream archives, but scarred by past bad experiences
good practices now could lead to partnerships in the future
these are living archives — not static.. still alive and growing
these ideas prompt re-evaluation of conventional archives thinking
lots of access to digital objects – perhaps movement to online existence

We need to understand that these communities evolve and are fluid. They have as broad variety of structures, sizes and methods of working. What are the patterns in participation & ownership?

The site urban 75 has hosted extended discussions about recent UK history. Efforts include identification of places and people in uploaded photos. The site connects people about issues about housing and local services – it is very practical but it also has evolved to include this historical documentation. One example post from the Brixton Forum shows a discussion about an Old shop front revealed on Atlantic Road.

A Short Aside

Next Flinn apologized for taking his talk slightly off script. Setting his papers aside, he spoke to the audience about the eXHulme website which he had discovered the evening before while finishing his presentation. Having lived in Hulme, Manchester himself, he felt a great impact from looking through the site. He spent 4 hours looking at it – including photos such as the travellers living in their buses parked – otteburn close 1996 seen at the bottom of this page. His discovery and exploration of this site gave him a greater personal understanding of the impact of these types of community documentation projects. I felt he would have been happy to keep talking about this site and the directions it had sent his thoughts — but he then got back to his papers and continued.

Building Community Online

Interactions online are the historic record of the community itself. Archives evolve and change as the community builds and edits their online content. These heritage and archive sites work to shift from the idea of visitors to engaging users in interaction — they need users of the website to feel part of the community.

Examples of sites building community online:

My Brighton and Hove – community history site
Remembering Olive Collective – “social production of collective knowledge”
The Newham Story — uses social tagging

How do you successfully encourage participation (rather than large number of passive observers) which is crucial to the success of these types of initiatives? Lurking without contributing is easy – even if joining requires action. The rate of uptake may correspond with the sense of ownership. Heritage projects might encourage and sustain such participation. See Elisa Giaccardi & Leysia Palen’s article – The Social Production of Heritage through Cross-media Interaction: Making Place for Place-making.

Suggestions

encourage conversation and treat all stories as having value – value every account
promote a sense of ownership once a story has been shared
allow for multiple ways to engage with and share content and memories
recognize and let users shift from observer to active member

Flinn’s Conclusions

What are the challenges and perils facing community archives? Lack of resources. People are doing these things in unsustainable ways
Why should we sustain independent community archives? Benefit to individuals, communities and broader society.
What can professional archivists do? Support and partnership with groups seeking this sort of partnership.

My Thoughts

The image I included above is from the Library of Congress’s Flickr Commons project. If you read through the comments on this photo you can see a diverse group of individuals come together to document the history of Sylvia Sweets Tea Room. This is just another example of the process of documentation being as interesting as the original image itself.

There is still so much to learn in the arena of building productive online communities. Archivists working through how to archive what online communities create will need to understand how the process of creation is documented via various software tools. As the techniques for encouraging participation evolve – archivists will need to evolve right along with them. I think it is interesting to envision archivists working in this space and supporting these types of communities — becoming as much the champions of the community itself as preservers of a community’s collaborative creations.

Image Credit: Flickr Commons Library of Congress: Sylvia Sweets Tea Room, corner of School and Main streets, Brockton, Mass

As is the case with all my session summaries from SAA2009, please accept my apologies in advance for any cases in which I misquote, overly simplify or miss points altogether in the post above. These sessions move fast and my main goal is to capture the core of the ideas presented and exchanged. Feel free to contact me about corrections to my summary either via comments on this post or via my contact form.

Archivists and New Technology: When Do The Records Matter?

June 6, 2009

Navigating the rapidly changing landscape of new technology is a major challenge for archivists. As quickly as new technologies come to market, people adopt them and use them to generate records. Businesses, non-profits and academic institutions constantly strive to find ways to be more efficient and to cut their budgets. New technology often offers the promise of cost reductions. In this age of constantly evolving software and technological innovation, how do archivists know when a new technology is important or established enough to take note of? When do the records generated by the latest and greatest technology matter enough to save?

Below I have include two diagrams that seek to illustrate the process of adopting new technology. I think they are both useful in aiding our thinking on this topic.

The first is the “Hype Cycle“, as proposed by analyst Jackie Fenn at Gartner Group. It breaks down the phases that new technologies move through as they progress from their initial concept through to broad acceptance in the marketplace. The generic version of the Hype Cycle diagram below is from the Wikipedia entry on hype cycle.

Each summer, Gartner comes out with a new update on Where Are We In The Hype Cycle?. Last summer, microblogging was just entering the ‘Peak of Inflated Expectations’, public virtual worlds were sliding down into the ‘Trough of Disillusionment’ and location aware applications were climbing back up the ‘Slope of Enlightenment’. There is even a book about it: Mastering the Hype Cycle: How to Choose the Right Innovation at the Right Time.

The other diagram is the Technology Adoption Lifecycle from Geoffrey Moore’s Crossing the Chasm. This perspective on the technology cycle is from the perspective of bringing new technology to market. How do you cross the chasm between early adopters and the general population?

Archivists need to consider new technology from two different perspectives. When to use it to further their own goals as archivists and when to address the need to preserve records being generated by new technology. A fair bit of attention has been focused on figuring out how to get archivists up to speed on new web technology. In August 2008, ArchivesNext posted about hunting for Web 2.0 related sessions at SAA2008 and Friends Told Me I Needed A Blog posted about SAA and the Hype Cycle shortly thereafter.

But how do we know when a technology is ‘important enough’ to start worrying about the records it generates? Do we focus our energy on technology that has crossed the chasm and been adopted by the ‘early majority’? Do we watch for signs of adoption by our target record creators?

I expect that the answer (such as there can be one answer!) will be community specific. As I learned in the 2007 SAA session about preserving digital records of the design community, waiting for a single clear technology or software leader to appear can lead to lost or inaccessible records. Archivists working with similar records already come together to support one another through round tables, mailing lists and conference sessions. I have noticed that I often find the most interesting presentations are those that discuss the challenges a specific user community is facing in preserving their digital records. The 2008 SAA session about hybrid analog/digital literary collections discussed issues related to digital records from authors. Those who worry about records captured in geographic information systems (GIS) were trying to sort out how to define a single GIS electronic record when last I dipped my toes into their corner of the world in the Fall of 2006.

It is not feasible to imagine archivists staying ahead of every new type of technology and attempting to design a method for archiving every possible type of digital records being created. What we can do is make it a priority for a designated archivist within every ‘vertical’ community (government, literary, architecture… etc) to keep their ear to the ground about the use of technology within that community. This could be a community of practice of its own. A group that shares info about the latest trends they are seeing while sharing their best practices for handling the latest types of records being seen.

The good news is that archivists aren’t the only ones who want to be able to preserve access to born digital records. Consider Twitter, which only provides easy access to recent tweets. A whole raft of third-party tools built to archive data from Twitter are already out there, answering the demand for a way to backup people’s tweets.

I don’t think archivists always have the luxury of waiting for technology to be adopted by the majority of people and to reach the ‘Plateau of Productivity’. If you are an archivist who works with a community that uses cutting edge technology, you owe it to your community to stay in the loop with how they do their work now. Just because most people don’t use a specific technology doesn’t mean that an individual community won’t pick it up and use to the exclusion of more common tools.

The design community mentioned above spoke of working with those creating the tools for their community to ensure easy archiving down the line. In our fast paced world of innovation, a subset of archivists need to stay involved with the current business practices of each vertical being archived. This group can work together to identify challenges, brainstorm solutions, build relationships with the technology communities and then disseminate best practices throughout the archives community. I did find a web page for the SAA’s Technology Best Practices Task Force and its document Managing Electronic Records and Assets: A Working Bibliography, but I think that I am imagining something more ongoing, more nimble and more tied into each of the major communities that archivists must support. Am I describing something that already exists?

NARA Outgrows ARC: Researching New Catalog Software Options

March 19, 2009 1 Comment

The Archival Research Catalog (ARC) of the US National Archives and Records Administration (NARA) needs to be replaced. NARA has put out an official Request for Information (RFI) and plans a “Vendor Day” for April 6th with final responses required by April 24th, 2009.

This is exciting for two very different reasons:

New catalog software!
Getting to read all the gory details about ARC!

If this makes you curious, then go give the RFI a read, but here are some juicy ARC tidbits to consider:

ARC’s Logical Data Model – 20 pages worth of data model that I am sorely tempted to print out, tape together and hang on a very large wall
ARC was built as a customization of OLIB back in 2003 and has been upgraded along the way
ARC currently contains 2,478,259 archival descriptions and 8,810,938 authority records
An average of 25,000 archival descriptions are added to ARC each week

The RFI states: “NARA has outgrown the existing ARC system and requires a more robust solution that’s capable of scaling to support at least 250 million archival descriptions and links to upwards of 500 million digital copies over the next 4-7 years.” Why so many records? Because all of NARA’s partners are digitizing records so quickly that they are creating a massive backlog of documents and the future only holds more of the same.

This RFI is only for planning purposes, but I will definitely be following this story as it unfolds.

ArchivesZ Data Challenges: Oregon State University Archives

February 22, 2009 1 Comment

The Oregon State University Archives has generously contributed 356 of their finding aids in EAD format for use in the development of version 2 of ArchivesZ. This is my first post in a what will likely be a series of looks behind the scenes at the challenges facing a project like ArchivesZ on the data level.

Version one of ArchivesZ only used finding aids from the University of Maryland and the Library of Congress. This was definitely a case of the path of least resistance. I attend the University of Maryland and the Library of Congress has a very convenient page providing links to all their Finding Aids source XML files. A very key aspect of creating version 2 of ArchivesZ is making sure that the scripts that pull data from EAD XML files is robust enough to handle the encoding practices of a very diverse range of institutions.

Please keep in mind that OSU is likely to bear the brunt of many basic data issues that I would have unearthed with whatever data sets I tried first!

There are 3 crucial data elements on which the visualizations of ArchivesZ depend: subject, inclusive dates, and collection size. Each element presents unique challenges. The script parsing issues I am uncovering with the OSU finding aids are currently worst for collection size. In order to make pretty charts which let people compare the quantity of materials in each collection (or record group – please forgive that I use the term ‘collection’ to mean any set of records for which a finding aid has been created), we need to be able to assign a single number to represent the size of each collection. Based on the values used in the LOC and UMD finding aids, we chose to go with linear of feet as our standard unit of measurement. So the trick is to translate whatever archivists choose to put into the <physdesc> element of their finding aid into some number of linear feet.

These are the size conversion rules we implemented for version 1 of ArchivesZ:

1 microfilm reel = 1 linear foot
Collections represented only by a number of items will be represented as .25 linear feet
If size only specified in number of boxes, then 1 box = .5 linear feet
When the size is given in some different types of units, they are prioritized in the following order: linear feet > boxes > microfilm reels > items

This works reasonably well when the physical description values are simple – it starts to fall apart when what is entered is more complicated. Here are some examples of the physical descriptions in the OSU finding aids:

Guide to the Phi Kappa Phi-OSU Chapter Records: The display in the ‘pretty’ version of the finding aid online shows this: 5.5 cubic feet (9 boxes, including 2 oversize boxes) (3 microfilm reels)

The version in the XML file is this:

<physdesc>
  <extent>5.5 cubic feet</extent>
  <extent>9 boxes, including 2 oversize boxes</extent>
  <extent>3 microfilm reels</extent>
</physdesc>

With the current algorithm, this finding aid would be marked as being 3 linear feet in size. At a bare minimum, I must add ‘cubic feet’ as another unit to be converted. More difficult to discern is if I should have a value of 5.5 linear feet (assuming 1 cubic foot = 1 linear foot for the purposes of these comparisons) or a value of 8.5 linear feet (5.5 + 3 linear feet for the 3 microfilm reels). There is never going to be a perfect answer here, but clearly my logic needs to be more sophisticated than it is now.

Harvey L. McAlister Collection: The display in the pretty version of this finding aid online is this: 1 cubic foot, including 26 photographs (4 boxes, including 2 oversize boxes, and 1 map folder)

The version in the XML file is this:

<physdesc>
  <extent encodinganalog="300$a">1 cubic foot, including 26 photographs</extent>
  <extent encodinganalog="300$a">4 boxes, including 2 oversize boxes, and 1 map folder</extent>
</physdesc>

With the current algorithm, this finding aid would be marked as being 1 linear foot in size. From looking at these two examples, it would seem that this would be fine and in fact – for the purposes of calculating a comparable size – only looking at the first <extent> value might be the way to go – at least for OSU finding aids.

There are some other simpler issues relating to standardization in the way that certain values are entered. For example, after ingesting 173 finding aids from OSU (the number I got through before my script flat out choked on a size designation), I ended up with five different repositories added to my REPOSITORIES table. I had expected only one. Each of these was entered as repository name — and I have included the length of each value to show how extra spaces are causing part of the problem:

Oregon State University Libraries – length 36
Oregon State University – length 23
Oregon State UniversityLibraries – length 32
Oregon State University Libraries – length 36
Oregon State University Libraries – length 33

Some of these I can handle by adding smarter trimming of trailing spaces – but in this case it is clear that typos and inconsistency are also a challenge. I checked and each of these different <corpname> values, within the <repository> element is used by at least 10 finding aids. Perhaps they have been inherited over time from a template?

I have considered creating a repository definition file that could be used when loading finding aids from one repository at a time. This would remove dependence on perfect replication of these sorts of values while still supplying the data needed to let people limit their searches by a named repository.

The last issue is the most minor. There are many /n and /t characters throughout the XML documents. These I plan to simply strip out as the script parses the XML file.

A big thank you to Elizabeth Nielsen, Senior Staff Archivist at OSU Archives. Her response to my query about OSU’s comfort with my taking apart their finding aids in public on my blog was “Bring it on – we’re tough!”.

It is fascinating to dig into new finding aids and see how the parsing script handles what it finds. I plan to test the existing script on XML from more sources to see all the things that must be fixed. Then I get to wrap my head around code that someone else wrote (another member of the original ArchivesZ team wrote the version 1 ruby script). For those of you who are not programmers, you can skim through my Book Review of Dreaming in Code to get a handle on why this can be harder than it sounds like it should be.

Want to share your institution’s EAD finding aids in XML format with the ArchivesZ project? Please drop me a line via my contact form.

Image Credit: OSU Archives image above from the OSU Archives Home Page.

SAA2009: Building, Managing and Participating in Online Communities

December 12, 2008 5 Comments

SAA 2009: Sustainable Archives AUSTIN 09 It is official – the panel I proposed for SAA 2009 (aka, Sustainable Archives: AUSTIN 2009) was accepted!

Title: Building, Managing and Participating in Online Communities: Avoiding Culture Shock Online

Abstract: As more archival materials move online, archivists must become adept at participating in and managing online communities. This session will discuss real world experiences of this involvement, including putting images into the Flickr Commons and links to archival materials in Wikipedia, as well as guidelines on cultural norms within online communities. We will also discuss choosing between building new communities from scratch vs joining a broader, existing community (such as the Flickr Commons).

I will be serving as session chair and moderator for our group of fabulous panelists (finances and travel plans permitting):

Deborah Wythe: talking about Flickr Commons and other Brooklyn Museum web/community projects (or whatever the latest and greatest projects are afoot at the Brooklyn Museum by the time we hit August 2009)
Ann Lally: talking about Wikipedia and blogs (co-author of: Using Wikipedia to Extend Digital Collections)
Mark Matienzo: talking about NYPL web/community projects
Seb Chan: talking about Powerhouse Museum, Flickr Commons and (maybe) blogs

The intention is for this session to begin with very brief presentations showing off the current projects at our panelists’ institutions and follow that up with lots of time for discussion and answering of questions.

We see our target audience as archivists who want to hear about real world experiences of working within existing online communities (such as Wikipedia or Flickr) and building new communities dedicated to cultural heritage materials. The session will target individuals with less experience with Web 2.0 and social media implementations, but the lessons learned should also be of interest to those already in the implementation stages of their own projects.

I will put out a call for questions as we get closer to the conference so that our group can get an idea of what people are interested in learning about specifically, so start making notes now. Hope to see you in Austin!

Susa 2.0: Max Evans’ Finding Aid Prototype

December 8, 2008 4 Comments

Susa Young Gates As part of his portion of our SAA 2008 panel in San Francisco, Max Evans demonstrated his prototype for a new way to view an EAD finding aid. You can download his presentation from the SAA’s site: Finding Aids for the 21st Century: The Next Evolution.

Max’s prototype of Susa 2.0 is now online! He asked that I make sure you know it works best (showing all the intended mouse over text for links) with Internet Explorer version 6.0. The prototype presents the finding aid of the Susa Young Gates Papers from the Utah State Historical Society. His design tackles the major issues that plague large finding aids normally displayed in traditional single page layouts. Anyone who has looked at a large finding aid online has had the experience of being scrolled down somewhere in the middle and realizing they have no idea what they are looking at. What folder is this item in? What box is this folder in? Am I reading through a list of letters from 1950 or are these the ones from 1970?

Context is hard to communicate when you are dealing with long lists of folders that stretch longer than the length of the screen. Max’s design uses a three column approach to provide context from left to right. His design also gives users a way to look at the full list of either items or folders, independent of their originating containers – each list then sortable in three different ways: ‘as arranged’, alphabetically or by date. I love this page which shows how a scanned document might be displayed within the proper context of the collection – in this case, page 2 of document 1 of the General Correspondence from 1886-1909. All of these ideas get at the heart of giving researchers more control over how to tackle the records in a collection while making sure that they don’t loose the tools that ordered documents in a folder would provide them in the research room.

His prototype takes a step beyond just changing how the finding aid itself is presented – but also considers how the work flow of a researcher can be improved while also simplifying the record request processes. The prototype gives the patron the option to request the scanning of specific folders or items. They can also add records to their ‘research cart’ to either request the proper boxes be retrieved or to store the records in a personal research area within the archives website – both possibilities sound useful to me.

Max’s prototype is such a great example of rethinking how people are expected to work with archival records within the confines of the information we already have available in finding aids as they exist today. I highly recommend you give Susa 2.0 a look. It is a testament to Max’s incredible patience that he was able to create this prototype using over 200 separate HTML files – but it also sets the bar high for what we could be doing with our interface design!

NEH Digital Humanities Startup Grant News: Visualizing Archival Collections

September 12, 2008 2 Comments

As of August 22nd, 2008 it was official. There is even a blog post over on the NEH Office of Digital Humanities updates page to prove it. The University of Maryland was granted a Level I NEH Digital Humanities Startup Grant to fund work on the ‘Visualizing Archival Collections’ project. The official one liner is that the project will support “The development of visualization tools for assessing information contained in electronic archival finding aids created with Encoded Archival Description (EAD)”. Why did I wait so long to announce this on the blog? I wanted to have something fun to announce at the end of my SAA presentation out in San Francisco!

The project director is Dr. Jennifer Golbeck. I also have the support of University of Maryland’s Jennie Levine, Dr. Bruce Ambacher, and Dr. Doug Oard. This amazing set collaborators should help me stay on the right track and make sure I keep the sometimes competing issues relating to archives, information retrieval and interface design in balance.

I will be collecting EAD encoded finding aids over the next few months. My goal is to gather a broad sample of English language finding aids from a wide range of institutions and work on the script that extracts this data into a database. Once we have the data extracted I get to look at what we have, do some data cleanup and start thinking about what sorts of visualizations might work with our real world data. During the spring term we will design and build a 2nd generation prototype of ArchivesZ.

Want your data to be part of this? If you would like to contribute EAD finding aids in XML format to the project, please send me the following information:

Archives Name
Archives Parent Institution (if applicable)
Archives Location
Contact at Archives for questions about the finding aids (name, email and phone number)
Estimate of # of finding aids being offered
Controlled Vocabulary or Thesaurus used for Subject values (as many as are used)
Method of finding aid delivery (sending me a zip file? pointing me at a directory online? some other way?)
Do I have your permission to post a discussion of the data issues I may find in your finding aids here on Spellbound Blog? (Please see the OSU Archives post as an example of they types of issues I discuss)

You can either put this into the form on my Contact Page or send email directly to jeanne AT spellboundblog dot com.

Thank you to everyone for their enthusiasm about the ArchivesZ project. It is very exciting to have the opportunity to take all these shiny ideas to the next level.

SAA2008: Preservation and Experimentation with Analog/Digital Hybrid Literary Collections (Session 203)

September 6, 2008

The official title of Session 203 was Getting Our Hands Dirty (and Liking It): Case Studies in Archiving Digital Manuscripts. The session chair, Catherine Stollar Peters from the New York State Archives and Records Administration, opened the session with a high level discussion of the “Theoretical Foundations of Archiving Digital Manuscripts”. The focus of this panel was preserving hybrid collections of born digital and paper based literary records. The goal was to review new ways to apply archival techniques to digital records. The presenters were all archivists without IT backgrounds who are building on others work … and experimenting. She also mentioned that this also impacts researchers, historians, and journalists.For each of the presenters, I have listed below the top challenges and recommendations. If you attended the sessions, you can skip forward to my thoughts.

Norman Mailer’s Electronic Records

Speaker: Gabriela Redwine from University of Texas at Austin’s Harry Ransom Center
Featured Collection: Norman Mailer Papers

Challenges & Questions:

3 laptops and nearly 400 disks of correspondence
While the letters might have been dictated or drafted by Mailer, all the typing, organization and revisions done on the computer were done by his assistant Judith McNally. This brings into question issues of who should be identified as the record creator. How do they represent the interaction between Mailer & McNally? Who is the creator? Co-Creators?
All the laptops and disks were held by Judith McNally. When she died all of her possessions were seized by county officials. All the disks from her apartment were eventually recovered over a year later – but it causes issues of provenance. There is no way to know who might have viewed/changed the records.

Revelations and Recommendations:

What is accessioning and processing when dealing with electronic records? What needs to be done?

gain custody
gather information about creator’s (or creators’) use of the electronic records. In March 2007 they interviewed Mailer to understand the process of how they worked together. They learned that the computers were entirely McNally’s domain.
number disks, computers (given letters), other digital media
create disk catalog – to reflect physical information of the disk. Include color of ink.. underlining..etc. At this point the disk has never been put into a computer. This captures visual & spacial information
gather this info from each disk: file types, directory structure & file names

The ideal for future collections of this type is archivist involvement earlier – the earlier the better.

Papers of Peter Ganick

Speaker: Melissa Watterworth
Featured Collection: Papers of Writer and Small Press Publisher Peter Ganick, Thomas J Dodd Research Center, University of Connecticut

Challenges & Questions:

What are the primary sources of our modern world?
How do we acquire and preserve born digital records as trusted custodians?
How do we preserve participatory media – maybe we can learn from those who work on performance art?
How do we incrementally build our collections of electronic records? Should we be preserving the tools?
Timing of acquisition: How actively should we be pursuing personal archives? How can we build trust with creators and get them to understand the challenges?
Personal papers are very contextual – order matters. Does this hold true for born digital personal archives? What does the networking aspect of electronic records mean – how does it impact the idea of order?
First attempt to accession one of Peter Ganick’s laptops and the archivist found nothing she could identify as files.. she found fragments of text – hypertext work and lots of files that had questionable provenance (downloaded from a mailing list? his creations?). She had to sit down next to him and learn about how he worked.
He didn’t understand at first what her challenges were. He could get his head around the idea of metadata and issues of authenticity. He had trouble understanding what she was trying to collect.
How do we arrange and keep context in an online environment?
Biggest tech challenge: are we holding on for too long to ideas of original order and context?
Is there a greater challenge in collecting earlier in the cycle? What if the creator puts restrictions on groupings or chooses to withdraw them?
Do we want to create contracts with donors? Is that practical?

Revelations and Recommendations:

Collect materials that had high value as born digital works but were at a high risk of loss.
Build infrastructure to support preservation of born digital records.
Go back to the record creator to learn more about his creative process. They used to acquire records from Ganick every few years.. that wasn’t frequent enough. He was changing the tools he used and how he worked very quickly. She made sure to communicate that the past 30 years of policy wasn’t going to work anymore. It was going to have to evolve.
Created a ‘submission agreement’ about what kinds of records should be sent to the archive. He submitted them in groupings that made sense to him. She reviewed the records to make sure she understood what she was getting.
Considering using PDFa to capture snapshot of virtual texts.
Looked to model of ‘self archiving’ – common in the world of professors to do ongoing accruals.
What about ’embedded archivists’? There is a history of this in the performing arts and NGOs and it might be happening more and more.

George Whitmore Papers

Speaker: Michael Forstrom: Beinecke Rare Book and Manuscript Library, Yale University
Featured Collection: George Whitmore Papers

Challenges & Questions:

How do you establish identity in a way that is complete and uncorrupted? How do you know it is authentic? How do you make an authentic copy? Are these requirements as unreasonable and unachievable?

Revelations and Recommendations:

Refresh and replicate files on a regular schedule.
They have had good success using Quick View Plus to enable access to many common file formats. On the downside, it doesn’t support everything and since it is proprietary software there are no long term guarantees.
In some cases they had to send CP/M files to a 3rd party to have them converted into WordStar and have the ascii normalized.
Varied acquisition notes.. and accession records.. loan form with the 3rd party who did the conversion that summarized the request.. they did NOT provide information about what software was used to convert from CP/M to DOS. This would be good information to capture in the future.
Proposed an expansion of the standards to include how electronic records were migrated in the <processinfo> processing notes.

Questions & Answers

Question: As part of a writers community, what do we tell people who want to know what they can DO about their records. They want technical information.. they want to know what to keep. Current writers are aware they are creating their legacy.

Answer: Michael: The single best resource is the interPARES 2 Creator Guidelines. The Beineke has adapted them to distrubute to authors. Melissa: Go back to your collection development policies and make sure to include functions you are trying to document (like process.. distribution networks). Also communities of practice (acid free bits) are talking about formats and guidelines like that Gabriela: People often want to address ‘value’. Right now we don’t know how to evaluate the value of electronic drafts – it is up to authors.

Question: Cal Lee: Not a question so much as an idea: the world of digital forensics and security and the ‘order of volatility’ dictate that everyone should always be making a full disk copy bit by bit before doing anything else.

Comment: Comment on digital forensic tools – there is lots of historical and editing history of documents in the software… also delete files are still there.

Question: Have you seen examples of materials that are coming into the archive where the digital materials are working drafts for a final paper version? This is in contrast to others are electronic experiments.

Answer: Yes, they do think about this. It can effect arrangement and how the records are described. The formats also impact how things are preserved.

Question: Access issues? Are you letting people link to them from the finding aids? How are the documents authenticity protected.

Answer: DSpace gives you a new version anytime you want it (the original bitstream) .. lots of cross linking supports people finding things from more than one path. In some cases documents (even electronic) can only be accessed from within the on site reading room.

Question: What is your relationship is like with your IT folks?

Answer: Gabriela: Our staff has been very helpful. We use ‘legacy’ machines to access our content. They build us computers. They are also not archivists, so there is a little divide about priorities and the kind of information that I am interested in.. but it has been a very productive conversation.

Question: (For Melissa) Why didn’t you accept Peter’s email (Melissa had said they refused a submission of email from Peter because it didn’t have research value)?

Answer: The emails that included personal medical emails were rejected. The agreement with Peter didn’t include an option to selectively accept (or weed) what was given.

Question: In terms of gathering information from the creators.. do you recommend a formal/recorded interview? Or a more informal arrangement in which you can contact them anytime on an ongoing basis?

Answer: Melissa: We do have more formal methods – ‘documentation study’ style approaches. We might do literature reviews.. Ultimately the submission agreement is the most formal document we have. Gabriela: It depends on what the author is open to.. formal documentation is best.. but if they aren’t willing to be recorded, then you take what you can get!

My Thoughts

I am very curious to see how best practices evolve in this arena. I wonder how stories written using something like Google Documents, which auto-saves and preserves all versions for future examination, will impact how scholars choose to evaluate the evolution of documents. There have already been interesting examinations of the evolution of collaborative documents. Consider this visual overview of the updates to the Wikipedia entry for Sarah Palin created by Dan Cohen and discussed in his blog post Sarah Palin, Crowdsourced. Another great example of this type of visual experience of a document being modified was linked to in the comments of that post: Heavy Metal Umlaut: The Movie. If you haven’t seen this before – take a few minutes to click through and watch the screencast which actually lets you watch as a Wikipedia page is modified over time.

While I can imagine that there will be many things to sort out if we try to start keeping these incredibly frequent snapshot save logs (disk space? quantity of versions? authenticity? author preferences to protect the unpolished versions of their work?) – I still think that being able to watch the creative process this way will still be valuable in some situations. I also believe that over time new tools will be created to automate the generation of document evolution visualization and movies (like the two I link to above) that make it easy for researchers to harness this sort of information.

Perhaps there will be ways for archivists to keep only certain parts of the auto-save versioning. I can imagine an author who does not want anyone to see early drafts of their writing (as is apparently also the case with architects and early drafts of their designs) – but who might be willing for the frequency of updates to be stored. This would let researchers at least understand the rhythm of the writing – if not the low level details of what was being changed.

I love the photo I found for the top of this post. I admit to still having stacks of 3 1/2 floppy disks. I have email from the early days of BITNET. I have poems, unfinished stories, old resumes and SQL scripts. For the moment my disks live in a box on the shelf labeled ‘Old Media’. Lucky me – I at least still have a computer with a floppy drive that can read them!

Image Credit: oh messy disks by Blude via flickr.

As is the case with all my session summaries from SAA2008, please accept my apologies in advance for any cases in which I misquote, overly simplify or miss points altogether in the post above. These sessions move fast and my main goal is to capture the core of the ideas presented and exchanged. Feel free to contact me about corrections to my summary either via comments on this post or via my contact form.

Freebase Parallax Search Interface: Exploring Olympic Games Facts

August 16, 2008 3 Comments

Well-formed data posted about a new Freebase project named Parallax. This new search interface takes faceted browsing another step – in this case making it easy to jump sideways from one dataset to another related dataset. Parallax still includes filters on the left side – but the twist comes from the opportunity to select what are called ‘Connections’ from the list in the upper right hand corner of the search results page.

This sort of thing makes the most sense when you can see examples. The creator of Parallax has published a great little video tour, but I also wanted to show you some neat data sets that were very easy to discover and embed in my blog. Since so many people are thinking about the Olympics right now, I thought I would start by exploring the Olympic Games Collection from Freebase. Below I have two data sets. On the left you will see a list of Olympic Games – and on the right you will see a list of Olympic event venues. (NOTE: to those reading this through a feed reader – you will likely have to click through to view the lists)

Now lets take a real sidestep and pull up a list of sports teams who use a former Olympic facility as a venue. This is the sort of question that you could figure out on your own, but it would be a pain in the neck to do by hand. See the list on the left below which took just as long to create as it took me to spot that Team (venue) was on the list of ‘more connections’ when my list of Olympic Venues was being displayed. The frame on the right below displays the one Olympic Venue that Freebase knows to have won an award (in this case the Structural Special Award).

Of course the lists above are only as good as the data behind them, but you can see how interesting it could be to use Parallax to explore connected information. Now take this idea to the world of archives and libraries, OPACs and finding aids and imagine the sorts of questions you can start asking. Yes – it does depend on the data being connected, but that is happening more and more all the time. The promise of the semantic web is structured data everywhere we turn.

Go play with Parallax. Look at Venture Funded Companies and then look at all the Games Developed by those companies. Examine the list of Bird Species and then see what schools have bird mascots… and THEN see a list of famous people who went to schools that have bird mascots.

Put in your own search from the Parallax homepage and play with the available connections. Map and timeline views are also available – though they only work if your data includes location and temporal data, respectively. If you find a great sequence of data sets – please share them!

Category: software

My Thoughts