Menu Close

Category: interface design

ArchivesZ Data Challenges: Utah Government Archives & Records Service

6 Comments

Utah dot Gov LogoGina Strack of the Utah State Archives and Records Service provided me with access to the XML of 1,196 EAD encoded finding aids. These EAD 2.0 XML files are a product of a grant funded project completed last year to migrate from EAD 1.0 finding aids. Their website includes a detailed account of the EAD Project.

These finding aids have helped me identify three types of ArchivesZ data challenges:

  • strange characters
  • broad composite subjects
  • determination of accurate collection size

Strange and mysterious characters!

These finding aids use a special character in the place of the standard Library of Congress double dash which normally appears between subsections of the subject heading.

An example subject from the Utah Government XML looks like this:

Women—Suffrage—Utah.

Viewing the same subject in a pure text editor (such as vi):

Women—Suffrage—Utah.

By the time it gets into my database and is pulled out via a query in MySQL Query Browser it looks like this:

Women?¢‚Ǩ‚ÄùSuffrage?¢‚Ǩ‚ÄùUtah.

Rather than just stripping out all instances of —,  my plan is to replace them with the standard Library of Congress double dash. This will ensure that the existing code that breaks the subjects down to tags will still work.

Composite Subjects

When I say “composite subject” what I mean is a subject that includes multiple very disparate terms. Rather than the Library of Congress style subjects, all aspects of which relate to the collection in question, these composite subjects cover multiple subjects which are grouped together for convenience.

This is a list of some of the most popular subjects for the Utah Gov collections:

  • Politics, Government, and Law
  • Business, Industry, Labor, and Commerce
  • Science, Technology, and Health
  • Arts, Humanities, and Social Sciences

These subjects throw a monkey wrench into my theories about decomposing subjects based on commas. The collections to which these subjects are assigned likely fit in only one of the component themes. For example, the “Inventory of Publications from Department of Technology Services, 1993-2008” is assigned the subject “Science, Technology, and Health”. If I divide this subject into 3 separate tags, the Science and Health tags would be quite misleading.

So that leaves me a bit trapped. If I want to divide subjects such as “Art, Cuban, 20th century”, as I discuss in my Syracuse University post, then I end up also dividing these umbrella subjects which separate such very divergent terms with commas.

This issue goes on my list of reasons to add a repository configuration file for use by the data extraction script.

Accurate Collection Size

In my quest to convert all sizes to linear feet – sizes such as these are challenging:

  • 0.20 cubic foot and 1 microfilm reel
  • 0.35 cubic foot and 2 microfilm reels

I also have situations of sizes be specified in multiple sections of the finding aid. The Inventory of ALERT Foundation records from Governor Bangerter, 1986-1991 has a collection level size of “0.50 cubic foot and 2 microfilm reels”, but further down in this finding aid I see this:

series: ALERT Foundation records

  • box 1, folder 1: Documentary: “”Letters from our Children,”” Motion picture film reel, 16mm
  • box 1, folder 2: Documentary: “”Letters from our Children,”” VHS videocassette
  • box 1, folder 3: Documentary: “”Letters from our Children,”” VHS videocassette
  • box 1, folder 4: Documentary: “”Letters from our Children,”” VHS videocassette

When they said 2 microfilm reels – do they really mean a 16mm motion picture film reel and a VHS videocassette? Is there 1 VHS videocassette or 3? How sizes are specified in a specific repository’s finding aids is another possible candidate for a repository level configuration script.

Tagging Statistics

Finally, here are a few tag stats:

  • Only 31 tags (1.5% of all Utah Government tags) are associated with 10 or more collections
  • 1404 tags (71.5%) are assigned to only a single collection
  • 107 collections have been assigned only 1 tag
  • 10 collections have no subjects

Of course these statistics are based on the current incarnation of the data extraction script. After I modify the script, there will be a greater number of tags and (hopefully) more overlap of tags across multiple collections. These types of statistics should help me gauge how well my data extraction logic is working.

Read more →

SAA2009: Building, Managing and Participating in Online Communities

5 Comments

SAA 2009: Sustainable Archives AUSTIN 09It is official – the panel I proposed for SAA 2009 (aka, Sustainable Archives: AUSTIN 2009) was accepted!

Title: Building, Managing and Participating in Online Communities: Avoiding Culture Shock Online

Abstract: As more archival materials move online, archivists must become adept at participating in and managing online communities. This session will discuss real world experiences of this involvement, including putting images into the Flickr Commons and links to archival materials in Wikipedia, as well as guidelines on cultural norms within online communities. We will also discuss choosing between building new communities from scratch vs joining a broader, existing community (such as the Flickr Commons).

I will be serving as session chair and moderator for our group of fabulous panelists (finances and travel plans permitting):

The intention is for this session to begin with very brief presentations showing off the current projects at our panelists’ institutions and follow that up with lots of time for discussion and answering of questions.

We see our target audience as archivists who want to hear about real world experiences of working within existing online communities (such as Wikipedia or Flickr) and building new communities dedicated to cultural heritage materials. The session will target individuals with less experience with Web 2.0 and social media implementations, but the lessons learned should also be of interest to those already in the implementation stages of their own projects.

I will put out a call for questions as we get closer to the conference so that our group can get an idea of what people are interested in learning about specifically, so start making notes now. Hope to see you in Austin!

Read more →

Google Tackles Magazine Archives

13 Comments

Google Book Search: Popular Mechanics Jan 1905 Cover ImageAs has been reported around the web today, Google is now digitizing and adding magazines to Google Book Search. This follows on the tails of the recent Google Life Photo archive announcement.

I took a look around to see what I could see. I was intrigued by the fact that I couldn’t see a list of all the magazines in their collection. So I went after the information the hard way and kept reloading the Google Book Search home page until I didn’t see any new titles displayed in their highlighted magazine section. This is what I came up with, roughly grouped by general topic groupings.

Science and technology:

Lifestyle and city themed:

African American:

  • Ebony Jr!: May 1973 through October 1985
  • Jet: November 1961 through October 2008
  • Black Digest: Named ‘Negro Digest’ from November 1961 through April 1970, then Black Digest from May 1970 through April 1976.

Health, nutrition and organic:

  • Women’s Health and Men’s Health: January 2006 through present. I found it very amusing to be able to scan the covers of all the issues so easily – true for all of these magazines of course, but funny to see cover after cover of almost identically clad men and women exercising.
  • Prevention: January 2006 through the present
  • Better Nutrition: January 1999 through December 2004
  • Organic Gardening: November 2005 to the present
  • Vegetarian Times: March1981 through November 2004

Sports and the outdoors:

They of course promise more magazines on the way, so if you are reading this long after mid December 2008  I would assume there are more magazines and more issues available now. I hope that they make it easier to browse just magazines. Once they have a broader array of titles – how neat would it be to build a virtual news stand for a specific week in history? Shouldn’t be hard – they have all the metadata and cover images they need.

I love being able to read the magazine – advertising and all. They display the covers in batches by decade or 5 year period depending on the number of issues. I also like the Google map provided on each magazines ‘about’ page that shows ‘Places mentioned in this magazine’ and easily links you directly to the article that mentions the location marked on the map.

I think it is interesting that Google went with more of a PDF single scrolling model rather than an interface that mimics turning pages. In many issues (maybe all?) they have hot-linked the table of contents so that you can scroll down to that section instantly. You can also search within the magazine, though from my short experiments it seems that only the articles are text indexed and the advertisements are not.

Google’s current model for search is to return results for magazines mixed in with books in Google Book Search results – but they do let you limit your results to only magazines from their Advanced Search page within Google Book Search. See these results for a quick search on sunscreen in magazines.

Overall I mark this as a really nice step forward in access to old magazines. As with many visualizations, seeing the about page for any of these magazines made me ask myself new questions.  It will be interesting to see how many magazines sign on to be included and how the interface evolves.

To read more about Google’s foray into magazine digitization and search take a look at:

For a really nice analysis of the information that Google provides on the magazine pages see Search Engine Land’s Google Book Search Puts Magazines Online.

Read more →

Susa 2.0: Max Evans’ Finding Aid Prototype

4 Comments

Susa Young GatesAs part of his portion of our SAA 2008 panel in San Francisco, Max Evans demonstrated his prototype for a new way to view an EAD finding aid. You can download his presentation from the SAA’s site: Finding Aids for the 21st Century: The Next Evolution.

Max’s prototype of Susa 2.0 is now online! He asked that I make sure you know it works best (showing all the intended mouse over text for links) with Internet Explorer version 6.0. The prototype presents the finding aid of the Susa Young Gates Papers from the Utah State Historical Society. His design tackles the major issues that plague large finding aids normally displayed in traditional single page layouts. Anyone who has looked at a large finding aid online has had the experience of being scrolled down somewhere in the middle and realizing they have no idea what they are looking at. What folder is this item in? What box is this folder in? Am I reading through a list of letters from 1950 or are these the ones from 1970?

Context is hard to communicate when you are dealing with long lists of folders that stretch longer than the length of the screen. Max’s design uses a three column approach to provide context from left to right. His design also gives users a way to look at the full list of either items or folders, independent of their originating containers – each list then sortable in three different ways: ‘as arranged’, alphabetically or by date. I love this page which shows how a scanned document might be displayed within the proper context of the collection – in this case, page 2 of document 1 of the General Correspondence from 1886-1909. All of these ideas get at the heart of giving researchers more control over how to tackle the records in a collection while making sure that they don’t loose the tools that ordered documents in a folder would provide them in the research room.

His prototype takes a step beyond just changing how the finding aid itself is presented – but also considers how the work flow of a researcher can be improved while also simplifying the record request processes. The prototype gives the patron the option to request the scanning of specific folders or items. They can also add records to their ‘research cart’ to either request the proper boxes be retrieved or to store the records in a personal research area within the archives website – both possibilities sound useful to me.

Max’s prototype is such a great example of rethinking how people are expected to work with archival records within the confines of the information we already have available in finding aids as they exist today. I highly recommend you give Susa 2.0 a look. It is a testament to Max’s incredible patience that he was able to create this prototype using over 200 separate HTML files – but it also sets the bar high for what we could be doing with our interface design!

Read more →

Political Campaign Ads from the NBC News Archives Find New Audience on Hulu.com

2 Comments

Thinking about politics, but waxing nostalgic for the good old days of movie stars and snappy jingles? Surf over to Hulu.com’s new gallery of Historic Campaign Ads. These are from iCue, which bills itself as “A fun, innovative learning environment built around the video from the NBC News Archives“.

And what would a political video blog post be without a political video? If you don’t see the video below, you can click through to view the I Like Ike ad from 1954 I chose for your viewing pleasure.

This is a great example of finding new audiences for material from archives. In this case, I had to dig for a while to discover that these were from the NBC News Archives. The Hulu iCue network/studio home page doesn’t really tell me anything – but you can imagine using a page like this to supply more information if you wanted to stress the archival origin of a set of videos.

Read more →

Freebase Parallax Search Interface: Exploring Olympic Games Facts

3 Comments

Well-formed data posted about a new Freebase project named Parallax. This new search interface takes faceted browsing another step – in this case making it easy to jump sideways from one dataset to another related dataset. Parallax still includes filters on the left side – but the twist comes from the opportunity to select what are called ‘Connections’ from the list in the upper right hand corner of the search results page.

This sort of thing makes the most sense when you can see examples. The creator of Parallax has published a great little video tour, but I also wanted to show you some neat data sets that were very easy to discover and embed in my blog. Since so many people are thinking about the Olympics right now, I thought I would start by exploring the Olympic Games Collection from Freebase. Below I have two data sets. On the left you will see a list of Olympic Games – and on the right you will see a list of Olympic event venues. (NOTE: to those reading this through a feed reader – you will likely have to click through to view the lists)

Now lets take a real sidestep and pull up a list of sports teams who use a former Olympic facility as a venue. This is the sort of question that you could figure out on your own, but it would be a pain in the neck to do by hand. See the list on the left below which took just as long to create as it took me to spot that Team (venue) was on the list of ‘more connections’ when my list of Olympic Venues was being displayed. The frame on the right below displays the one Olympic Venue that Freebase knows to have won an award (in this case the Structural Special Award).

Of course the lists above are only as good as the data behind them, but you can see how interesting it could be to use Parallax to explore connected information. Now take this idea to the world of archives and libraries, OPACs and finding aids and imagine the sorts of questions you can start asking. Yes – it does depend on the data being connected, but that is happening more and more all the time. The promise of the semantic web is structured data everywhere we turn.

Go play with Parallax. Look at Venture Funded Companies and then look at all the Games Developed by those companies. Examine the list of  Bird Species and then see what schools have bird mascots… and THEN see a list of famous people who went to schools that have bird mascots.

Put in your own search from the Parallax homepage and play with the available connections.  Map and timeline views are also available – though they only work if your data includes location and temporal data, respectively. If you find a great sequence of data sets – please share them!

Read more →

Online Interactive U.S. Copyright Slider

Digital Copyright Slider

Remember when I posted about the Copyright Slider: Quick Easy Access to Copyright Laws and Guidelines? This was my last line in that post:

My next question is how hard would it be to make a slick flash version of this that could live online and be updated as copyright rules change?

Well, thanks to Digitization 101’s A digital version of the Copyright Slider post I discovered that exactly what I wished for now exists. Go take the Digital Copyright Slider for a spin.

The interface is clear and simple – they did a great job of taking advantage of the interactive medium to do things they couldn’t do on the paper slider. If it won’t disturb your neighbors, turn up the volume to hear the satisfying click each time you move the slider to a new scenario. Make sure you click on some of the *s to see more detailed information. Take note of the advice regarding more complicated scenarios as well as the links directly to documents detailing specific copyright laws.

I love that it has been licensed under the Creative Commons Attribution-Noncommercial-Share Alike license. The creators have included their contact information along with the idea that other institutions could host custom copies of the slider with their own copyright research contacts. The only downside I see to this is that if there are changes to US copyright law, it will take time for updates to a central copy of the slider to propagate to local customized copies.

The final question is how fast they can update the slider in the event of changes to copyright law – but we will have to wait on changes to the US copyright landscape before we can find that out!

Image Credit: Image above is taken directly from a screen shot of the Digital Copyright Slider.

Read more →

Dipity: Easy Hosted Timelines

3 Comments

Dipity LogoI discovered Dipity via the Reuters article An open-source timeline of the virtual world. The article discusses the creation of a Virtual Worlds Timeline on the Dipity website. Dipity lets anyone create an account and start building timelines. In the case of the Virtual Worlds Timeline, the creator chose to permit others to collaborate on the timeline. Dipity also provides four ways of viewing any timeline: a classic left to right scrolling view, a flipbook, a list and a map.

I chose to experiment by creating a timeline for Spellbound Blog. Dipity made this very easy – I just selected WordPress and provided my blog’s URL. This was supposed to grab my 20 most recent posts – but it seems to have taken 10 instead. I tried to provide a username/password so that Dipity could pull ‘more’ of my posts (they didn’t say how many – maybe all of them?). I couldn’t get it to work as of this writing – but if I figure it out you will see many more than 10 posts.

I particularly like the way they use the images I include in my posts in the various views. I also appreciate that you can read the full posts in-place without leaving the timeline interface. I assume this is because I publish my full articles to my RSS feed. It was also interesting to note that posts that mentioned a specific location put a marker on a map – both within the single post ‘event’ as well as the full map view.

Dipity also supports the streamlined addition of many other sources such as Flickr, Picasa, YouTube, Vimeo, Blogger, Tumblr, Pandora, Twitter and any RSS feed. They have also created some neat mashups. TimeTube uses your supplied phrase to query YouTube and generates a timeline based on the video creation dates. Tickr lets you generate an interactive timeline based on a keyword or user search of Flickr.

Why should archivists care? I always perk up anytime a new web service appears that makes it easy to present time and location sensitive information. I wrote a while ago about MIT’s SIMILE project and I like their Timeline software, but in some ways hosted services like Dipity throw the net wider. I particularly appreciate the opportunity for virtual collaboration that Dipity provides. Imagine if every online archives exhibit included a Dipity timeline? Dipity provides embed code for all the timelines. This means that it should be easy to both feature the timeline within an online exhibit and use the timeline as a way to attract a broader audience to your website.

There has been discussion in the past about creating custom GoogleMaps to show off archival records in a new and different way.  During THATCamp there was a lot of enthusiasm for timelines and maps as being two of the most accessible types of visualizations. By anchoring information in time and/or location it gives people a way to approach new information in a predictable way.

Most of my initial thoughts about how archives could use Dipity related to individual collections and exhibits – but what if an archive created one of these timelines and added an entry for every one of their collections. The map could be used if individual collections were from a single location. The timeline could let users see at a glance what time periods were the focus of collections within that archives. A link could be provided in each entry pointing to the online finding aid for each collection or record group

Dipity is still in working out the kinks of some of their services, but if this sounds at all interesting I encourage you to go take a look at a few fun examples:

And finally I have embedded the Internet Memes timeline below to give you a feel of what this looks like. Try clicking on any of the events that include a little film icon at the bottom edge and see how you can view the video right in place:

Image Credit:  I found and ‘borrowed’ the Dipity image above from Dipity’s About page.

Read more →

THATCamp 2008: Day 1 Dork Short Lightening Talks

2 Comments

lightningDuring lunch on the first day of THATCamp people volunteered to give lightning talks they called ‘Dork Shorts’. As we ate our lunch, a steady stream of folks paraded up to the podium and gave an elevator pitch length demo. These are the projects about which I managed to type URLs and some other info into my laptop. If you are looking for examples of inspirational and innovative work at the intersection of technology and the humanities – these are a great place to start!

Have more links to projects I missed including? Please add them in the comments below.

Image credit: Lightning by thenss (Christopher Cacho) via flickr

Read more →

THATCamp 2008: Crowdsourced Transcription and Collaborative Annotation

13 Comments

Free Pencils by zone41 on Flickr

The THATCamp session officially titled ‘Crowdsourcing’ on the schedule was actually aimed at discussing the intersection of crowdsourced transcription and collaborative annotation. The group was small – just six of us and Ben Brumfield got us going by giving us an overview of transcription software and projects:

  • The FamilySearch Indexing Project is an LDS church project put out by the FamilySearch Labs. Their goals: “Volunteers extract family history information from digital images of historical documents to create searchable indexes that assist everyone in finding their ancestors.”
  • The Manuscript Transcription Assistant is based at Worcester Polytechnic Institute (WPI) and is described as “a tool to assist transcribers in creating transcriptions, and incorporate meta-data about each image and transcription that can then be used to search through an electronic library of transcriptions”. I found mention in the FAQ of the desire to create a community so that “transcribers will be able to collaborate their work by rating the quality of other user’s transcriptions. By ranking the transcriptions, specific versions of transcriptions will emerge as an authority for that manuscript. ” Unfortunately, a lot of the links on that site are broken and my attempt to register gave me an error. It is not clear to me that this project is actually still active.
  • Soldier Studies is a website dedicated to posting transcriptions of civil war letters and diaries. This is not a tool for transcribing, but is clearly a repository targeting specifically transcriptions (see their Mission Statement for more information).
  • Oh No Robot is a comics transcription and search tool. It provides a page to find comics needing transcription and a great page to explain how transcription works on their site.

After examining what was out there, Ben concluded that what he wanted didn’t exist – so he started to build it himself. He gave us a demo of his “very beta” software. His goal is to build a web based tool to support collaborative manuscript transcription and annotation by individuals without a strong technical background. In its current (and private beta) state the software supports transcription, an innovative approach to linking individual words or phrases to collection defined subjects and some basic community tools to let his virtual team discuss transcription issues. Ben is working hard on the software – if you are interested in his project, definitely get in touch with him.

Travis Brown showed us his creation: eComma. eComma aims to “enable groups of students, scholars, or general readers to build collaborative commentaries on a text and to search, display, and share those commentaries online”. He showed us how users could tag or add comments on individual words or phrases of a loaded text. Take a look at the eComma page for Sonnet 18 by William Shakespeare. The words highlighted in blue are those which are tagged or have comments associated with them. If you highlight ‘the eye of heaven’ in line 5 you will see that it is tagged as a metaphor. Travis reported that he will have 2 other programmers working on eComma with him this summer and has his eye on improving some interface issues and adding a few more features.

We also talked about ways to display transcription. Elena Razlogova guided us over to the DoHistory website. There she showed us the Magic Lens interface. This interface displays the transcription of a handwritten diary page via a lens style overlay that you can move with your mouse. This reminded me of the Gilder Lehrman Battle Lines: Letters from America’s Wars interface that I found when doing research for my Communicating Context in Online Collections Poster. If you haven’t seen it before – go examine the page showing the transcription of (turn down your speaker if a reader’s voice will disturb those around you) Nathanael Green’s letter to Catherine Greene dated July 17, 1778.

While on the DoHistory site I also found the Try Your Hand At Transcribing page. This page shows the challenge of transcribing handwritten documents by giving you the chance to try it yourself and then lets you check your transcription with the click of a button.

We talked a bit about the technology behind eComma (forgive me Travis for not having enough details in my notes to explain your current architecture here) and the challenges inherent in wanting to annotate overlapping sets of words. Though he isn’t using it in the current implementation of eComma, Travis mentioned the Layered Markup Annotation Language (LMNL) which the tutorial page explains as:

…LMNL documents contain character data which is marked up using named and occasionally overlapping ranges. Ranges can have annotations, which can themselves be annotated and can have structured content. To support authoring, especially collaborative authoring, markup is namespaced and divided into layers, which might reflect different views on the text.

I can definitely see how LMNL might be an interesting framework for building transcription and annotation software.

Krissy O’Hare brought up the challenges of transcribing audio and video that she has faced working on oral history projects at Concordia University. This led to Travis (I think?) mentioning the Texas German Dialect Project (TGDP) and the CMU Sphinx Group Speech Recognition Engine. TGDP has an online archive of recorded interviews along with their transcriptions and translations. CMU Sphinx’s introduction explains that their software tools are targeted at expert users wanting to build speech-using applications.

This was a great session. The small group gave everyone a chance to contribute and take over the keyboard in order to show off their favorite sites. It was immediately after the Text Mining session, so our minds were already full of all the great things one could do with text once it is transcribed.

I am excited to watch the evolution of group transcription and annotation software. If you know of other transcription or annotation tools or projects – please post them to the comments.

Image credit: Free pencils by zone41 via flickr

As is the case with all my session summaries from THATCamp 2008, please accept my apologies in advance for any cases in which I misquote, overly simplify or miss points altogether in the post above. These sessions move fast and my main goal is to capture the core of the ideas presented and exchanged. Feel free to contact me about corrections to my summary either via comments on this post or via my contact form.

Read more →