Google Books and OpenLibrary APIs ================================= :author: Dan Scott :copyright: 2011 Laurentian University :backend: slidy :data-uri: :max-width: 45em :icons: :duration: 15 This talk is licensed under a http://creativecommons.org/licenses/by-sa/2.5/ca/[Creative Commons, Attribution, Share Alike license] and is available from http://bzr.coffeecode.net/2011_olita . image::images/cc_by_sa_360.png[] 'WARNING': This presentation contains (very little!) code. 'DISCLAIMER': I hold (very few!) Google shares. Google Books v1 API ------------------- * Released at Google I/O start of May 2011, based on the Discovery API service * AJAX and Data APIs will die December 2011 * REST calls, JSON data * Strict limits on unauthenticated access * Free authenticated access, but low default limits * Extensive documentation Google API Console ------------------ The https://code.google.com/apis/console[Google API Console] appears to be Google's one-stop shop for all of its future APIs: * Create API projects, then manage and report on: ** Authentication tokens ** API usage ** Enable and disable APIs ** Request an increased quota (default: 1000 queries/day, 1 qps/user) * Check the terms of service image::images/google_api_console.png[] A fast happy response --------------------- The API is fast, and Google's customer service can be as well. A few hours after submitting a request for a quota increase: image::images/google_quota_increase.png[] A word about authentication --------------------------- * Depending on the operation, either OAuth2 or append an API key as a GET param * Unless you're an OAuth wizard, use the Google Discovery API language libraries: ** Beta: *** Java *** Python ** Alpha: *** .Net *** PHP *** Ruby * As civilized people, we shall use Python for our examples. Authentication screenshot ------------------------- With the Python client, at least, the first time you run a script you are asked to open a URL in a browser to authorize access: image::images/google_oauth2_perm_request.png[] Search for matching volumes --------------------------- `GET https://www.googleapis.com/books/v1/volumes?q={search terms}` * "{search terms}": ** `AND` by default ** `-` for exclusion ** `"quote"` for exact phrase ** `intitle:`, `subject:`, `inauthor:`, `inpublisher:`, `isbn:` to scope search * Other parameters enable you to filter by visibility, limit by download format, paginate [source, python] ------------------------------------------------------------------------------- def vol_by_query(self, query): "Return the metadata for all results for a given query" results = self.service.volumes().list(q=query).execute() return results ------------------------------------------------------------------------------- Response for a successful search -------------------------------- [source,sh] ------------------------------------------------------------------------------- curl 'https://www.googleapis.com/books/v1/volumes?q=apache+derby' ------------------------------------------------------------------------------- [source,javascript] ------------------------------------------------------------------------------- include::google_books_response.json[] ------------------------------------------------------------------------------- Add a volume to a bookshelf --------------------------- 1. Create a bookshelf using the Google Books UI in a browser 2. Grant access via OAuth2 to your application 3. POST to `https://www.googleapis.com/books/v1/mylibrary/bookshelves/{shelf}/addVolume?volumeId={volumeId}` with OAuth2 authentication Where: * `{shelf}` is the shelf identifier * `{volumeId}` is the Google Books volume identifier [source, python] ------------------------------------------------------------------------------- def add_volume(self, shelf_id, vol_id): """Add a volume to a user's book shelf""" # Requires an empty body argument? Okay. res = self.service.mylibrary().bookshelves().addVolume( body='', volumeId=vol_id, shelf=shelf_id ).execute(self.auth_http) return res ------------------------------------------------------------------------------- Searching a bookshelf ---------------------- Not currently possible in v1 API. Wait, what? Help! * Posted a http://www.google.com/support/forum/p/booksearch-apis/thread?tid=3485fddbd4123a2c&hl=en[question in Google Books API forum] on May 21 * Asked for help on #googleapis IRC channel; friendly, but no authorities * Opened http://code.google.com/p/google-ajax-apis/issues/detail?id=587[bug #587] in google-ajax-apis bug tracker on May 26 * Sent email to Google Books product manager No response so far. A bit disappointing. But one can divine a business rationale: force people to land on the Google Books property on the web, collect more usage data, and (possibly) sell books. OpenLibrary Read API -------------------- * Released end of May 2011 * REST calls (GET a single URI), JSON data * No limits on API usage * No authentication required * Limited documentation Goal of the API --------------- * Find matching or similar editions based on identifiers * For each match, return metadata and URLs for reading online or "borrowing" (http://blog.openlibrary.org/2010/06/29/small-moves-open-library-integrates-digital-lending/[introduced June 2010]) image::images/openlibrary_read_online.png[] OpenLibrary Reader ------------------ image::images/openlibrary_reader.png[] OpenLibrary downloadable formats -------------------------------- image::images/openlibrary_reader_downloads.png[] OpenLibrary Read API summary ---------------------------- All requests are GET and use the following template: `http://openlibrary.org/api/volumes/brief/json/`'[identifier-list]' where the '[identifier-list]' is a pipe-delimited set of one or more identifiers: * ISBN - (`isbn:`) prefix * OCLC - (`oclc:`) prefix * LCCN - (`lccn:`) prefix * OpenLibrary ID - (`olid:`) prefix Results are returned in a JSON hash with keys that match your requested identifiers. Each identifier key contains a `records` hash and an `items` array for matching bibliographic metadata and electronic resources, respectively. OpenLibrary READ API example ---------------------------- [source,sh] ------------------------------------------------------------------------------- curl http://openlibrary.org/api/volumes/brief/json/isbn:9780596156718| ------------------------------------------------------------------------------- [source,javascript] ------------------------------------------------------------------------------- include::openlibrary_response.json[] ------------------------------------------------------------------------------- JSONP support ------------- For pure JavaScript usage, the JSONP convention is supported. Append `?callback=`'' and the data will be returned wrapped in a call to the function you specified in '' [source,sh] ------------------------------------------------------------------------------- curl http://openlibrary.org/api/volumes/brief/json/isbn:9780596156718?callback=parseMe ------------------------------------------------------------------------------- [source,javascript] ------------------------------------------------------------------------------- parseMe({ "isbn:9780596156718": {"records": {...}, "items": [...] } }); ------------------------------------------------------------------------------- Proxying requests for more borrowing, happier browsers ------------------------------------------------------ * OpenLibrary Read API currently supports HTTP access only; modern browsers get angry if you include an HTTP resource in an HTTPS page * Borrowing partners of OpenLibrary are identified by IP address, so off-site users using JavaScript calls directly to OpenLibrary may be missing out on potential electronic borrowing options * Solution: proxy the OpenLibrary requests through your server ** Enables you to offer secure access and offer a consistent "Borrow" experience to your users ** Bonus: also avoids JavaScript same origin policy hassles * Don't forget to proxy access to the borrowed resource as well! Useful resources ---------------- * Google Books v1 API ** https://code.google.com/apis/console[Google API Console] ** Documentation: *** https://code.google.com/apis/books/docs/v1/getting_started.html[Getting started] *** https://code.google.com/apis/books/docs/v1/using.html[Using the API] *** https://code.google.com/apis/books/docs/v1/reference.html[Reference] ** http://bzr.coffeecode.net/2011_olita/google_books.py[google_books.py] - Python script that demonstrates how to iterate over MARC records, find matching volumes by ISBN in Google Books, and add those volumes to a custom bookshelf * OpenLibrary Read API ** http://blog.openlibrary.org/2011/06/03/announcing-a-new-read-api/[Announcing the Read API] ** http://openlibrary.org/dev/docs/api/read[Read API developer documentation] ** http://coffeecode.net/archives/249-The-wonderful-new-OpenLibrary-Read-API-and-Evergreen-integration.html[Coffee|Code blog post] ** http://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/user/dbs/openlibrary-read-api-master[Evergreen branch with OpenLibrary Read API integration]