October 2001 ZIG Meeting

October 2-5, Boston Spa, UK

Minutes: October 4

Minutes provided by Dana Dietz, OCLC

Welcome (Bill Moen)

Registered implementors get opportunity to update registration via circulating copy; contact Ray to update if you weren't in attendance.

Introductions and Updates. See Introduction of Attendees and Statuses

Bill and Ray remind all to submit project updates and status for posting on maintenancy agency page

Agenda Review

ZNG discussion; lunch at 12:30 PM; continue ZNG if needed; Digital Library issues (Herbert van de Sampel could not attend meeting so some new speakers will be drafted); project briefings

ZNG

Overview (Ray Denenberg):

power point slides

• What is the future of Z39.50? The ZHTTP profile did not go over well. What is the ZIG? The ZIG decided at the last meeting that Z39.50 needed to move forward. May the best idea for the future win.
• ZNG began as ZML project with Pat Stevens, Poul Henrik, Ray Denenberg and Bill Moen, based on the premise that Z39.50 must either evolve or be replaced.
• Whatever the next generation of Z39.50 is, it must preserve the intellectual capital of Z39.50.
• Two paths are being pursued at the same time. A is the possibility of a new protocol. B is evolution of Z39.50. They are complimentary approaches being pursued independently.
• ZNG focuses on path B. Decouple Z39.50 from the underlying syntax and transport. Develop profiles with different syntaxes. Focus on ZML profile.
• ZNG is a proof-of-concept initiative that seeks to lower the barrier to implementation and recognizes the current importance of Z39.50.
• The result set name is a URI (in ZML, not so in ZNG); several services are excluded including init.
• At the June meeting, ZML became ZNG. It was not intended to be a recasting of Z39.50; it was intended to be search and retrieval for the web.
• Z39.50 features like result sets, explain, diagnostics, abstract access points and record schemas will all be kept.

Applications (Bill Moen):

• Convinced by Pat Stevens that focus on user services matters, therefore a different approach is needed.
• Libraries provide traditional services, access to external services and access to external information sources.
• Integrated access to a variety of resources is what’s needed. Can we enable this through something other than traditional Z39.50 to incorporate the non-library world?
• Pat Stevens summarizes use of Z39.50 in three categories:
  • libraries need Z39.50 to support cooperation and resource sharing among themselves.
  • Libraries also want to give integrated access to scientific information sources' and other network resources not under their direct control.
  • Libraries want to help users seeking information elsewhere on the Web. (This is the reverse of case 2 as the library is the provider and the other party is managing the service.)
• Pat thinks ZNG can help with all three cases, but particularly cases 2 and 3.

Architecture (Poul Henrik):

power point slides

• ZML was presented at the Dec 2000 ZIG meeting: Z39.50 services were almost unchanged; ZER encoding of Z39.50 ASN.1 defined PDU structures; SOAP Remote Procedure Calls (RPC) used to exchange Z39.50 request/response pairs; HTTP used as transport mechanism for SOAP RPC
• ZNG was presented in Spring 2001: new search and retrieval web service; Web Services Definition Language (WSDL) spec; messages (parameters) in XML; port types (procedure calls and parameters); SOAP RPC binding for special clients; URL binding for modern browsers (that can handle XSL stylesheets) therefore no client required
• ZNG SOAP: query string with optional reference to previous set response; SOAP request message with number of first requested record, et.c; structure of response and of records returned (schema reference) can be extended to other things; SOAP reply message with result set reference, number of hits, etc. plus array of (XML formatted) records
• ZNG URL: used with web browser without need for special clients; request parameters encoded in URL syntax with server location; response returned as XML structure with optional stylesheet reference (can control presentation of data unlike Z39.50); can be used via browser page with simple embedded script
• Summary: based on HTTP (SOAP) and XML; specified by WSDL; SOAP RPC alternative requires special SOAP clients or HTTP web gateway services; URL RPC alternative requires only browser and HTML web page with scripts or HTTP web gateway

Comment/Question:

Lennie: …

Not sure who responded: SOAP uses remote prcedure call mechanism, does not necessarily run on browser but does necessarily run on HTTP. It is expected that most will be standalone.

Vision (Ralph LeVan):

• June meeting was still really minor evolution of Z39.50.
• Ralph was determined never to have to write a server again; bolt an application to a commercial product that can handle thousands of simultaneous users, use a servelets model, don’t use special tools to look at data, etc.
• XML is fatter and requires more bandwidth and disk space but saves programmer time and therefore money; the priority for this project is ease for programmers.
• The project is XML and HTTP based.
• At a meeting at OCLC prior to the DC meeting, time was spent discussing this project. Head of development at OCLC wanted Ralph to “make explain work right.” He wanted to be able to point a client at a server and automatically configure it.
• The desire is to make Z39.50 simplified so it is explainable. Get rid of multiple databases at a server; each database should be a separate connection.
• Make the protocol simpler. CORBA for Z39.50 was talked about years ago … it was too hard because of conditional parameters.
• Command language queries are needed.
• Rich attributes are great for describing semantics of indexes but user is talked to in client about indexes and indexes also exist in databases … simple indexes are adequate.
• CQL will be richened to say “title” from X (Dublin Core, etc.) … attributes don’t have to be on the wire. DC:title could be what is sent.
• This should be stateless as much as possible. Result set names are basically equivalent to queries. They could be looked up in a query and be cached or not. Enough do want result set names though so this will be optional; it will be a local optimization to reexecute the query or cash.
• Search and retrieval will be merged; records will come back from a query … this may need to be decomposed if more complexities are introduced.
• The group in June was receptive to these requirements as outlined by Ralph. Nobody had advance notice of this.

Question/Comment:

Rob Sanderson: What type of data is assumed? Small bib data?

Ralph: No fixed model; not just marc-type bib data practice in HTTP would be pointers but model doesn’t prohibit use of actual data (point to .gif file rather than sending the actual .gif file).

Rob Sanderson: The application in an archive for huge records could be an issue of actual data is sent.

Ray: We’ll talk later about actual metadata formats.

Schema (Matthew Dovey):

power points slides

• WSDL spec; WSDL is web service description language
• Messages:
• Search retrieve request is a query in CQL, start record, max # you want, format for record and response.
• Result set name can be use (db changes could cause new result set to be returned for the same search at a later point in time); disagrees with Ralph on this.
• SOAP encoded or not (client or browser); both are options.
• Query is optional.

Question/Comment:

Joe: Can you define multiple options per port?

Matthew: Yes but this will just do one.

• Bind port
• URL à query = startrecord=
• Bind to SOAP
• SOAP/HTTP example
• encryption and authentication via HTTP
• response à schema spec’d in request.
• Diagnostic schema behaves as another record format.
• Status signified in codes; codes not yet decided.
• Multiple diagnostic sets aren’t done yet but would be if needed.

Question/Comment:

Mike: Your request specified max # of records, so you don’t get back all with a query?

Matthew: All or none can be specified as well as a set number.

Explain (Janifer Gatenby):

power point slides

§ Database discovery: UDDI directory

§ Database capabiltity: mechanism to search without query and an XML document is record in ZNG explain that tells you what you can do.

§ Lighter than Lite: compare ZNG explain to Z39.50 explain lite

CQL (Ralph LeVan):

• CCL with a little bit of baggage … some things Z39.50 does: how to do ranges unambiguously is still not figured out; phrase versus wordlist?; BNF too hard and not done yet.
• Is NOT an operator? Ralph means it as a binary operator therefore it’s “ANDNOT” but BNF will probably have “NOT” and “AND NOT” as synonyms.
• Within and near, adjacency, index ids, attribute set names all optional.
• Ralph hates completeness and position. A string match is what equals is all about and special characters can then be relevant if they need to be.
• Truncation: Left and right? Put this in explain so that it’s up to the engine to decide whether to support.

Question/Comment:

Joe: Fundamental issue is that going back to query string is a step back and brings in ambiguity.

Ralph: It’s not that much more ambiguous.

Joe: Doesn’t think it’s simpler.

Ralph: Easier to type and don’t need to special tools.

Paul Miller: Ambiguity places burden on writers of clients to deal with vendors with different interpretations.

Rob Bull: Users can use a query language; only danger is short cuts users will want.

Rob Sanderson: Why not have GUI interface with drop downs if you have explain … why have command language?

Ralph: Assume GUI will be used.

Sebastian Hammer: Concerned about attribute stuff moving into syntax … complexity for evolution.

Ralph: This is possible. Would be worse with broadcast searching. Mechanism allows different query if you want.

Mike Taylor: Ralph, defend this more vigorously … it’s not ambiguous.

Ralph: It’s intended for humans to type.

Lennie: RPN is strength – why not keep it? Why should the command language be more specialized than word plus word?

Ralph: This will be prepared for more languages.

Matthew: It is meant to be a well formed language and therefore not ambiguous. System to system query language with anticipation of UI. Last ZIG meeting spun off number of initiatives that will also be talked about later.

Ralph: Intent is that server doesn’t have to interpret. It will announce what it supports and should only be sent what it supports.

LUNCH

Look toward early summer of next year in the US for next meeting (who will host?) and in the UK after that (host?).

Implementation Experience (Ralph LeVan):

• Building SOAP server was hoped to be a nearly trivial task. Found a client by searching google, web development tools are open source and findable, but it was not trivial.
• It’s context dependent so where you put things on a disk matters and altering your file configuration means you have to massage the environment.
• The grief experienced will be applicable to the future; Z39.50 grief is not reusable.
• SOAP toolkit: simple RPCs easy to configure, no mechanism to specify init parameters (talked to listserv but got no advice) therefore extended SOAP servelet itself by overriding init method.
• Mechanisms are easy to develop … Ralph will write up what he did.
• This was not trivial but components are there.
• Got server, in process of bolting to database engine.
• Will talk with Ian about PQN query parser.
• Will have SOAP-based server by next week.

Implementation Experience (Matthew Dovey):

§ ZNG is an umbrella that embraces a number of experiments.

§ ZML

§ SOAP Search Service (with Eliot and Dave) is similar to ZNG but multiple databases, XML structure for query type, etc.

§ Rewrite of ASN.1 as WSCL (Web Service Conversation Language): SOAP WSCL toolkits; SOAP server tools written on assumption code and API exists; building ZNG gateway to Oxford.

Metadata (Ray Denenberg):

• DC
• ONIX (these two were considered in June. Now the following are considered in addition.)
• OAI MARC schema.
• METS (Metadata Encoding Transmission Standard: descriptive, administrative, and structural metadata for digital objects)
• “Red Sox” (no real name yet, but it is being developed; XML schema with human-interpreted element names corresponding to MARC fields).

General Discussion on ZNG:

Pieter: Pat Stevens says ZNG is “user to business and Z39.50 is “business to business.”

Ray: ZNG is good for the library as broker for external-library information and for the accessing of library resources by external organizations; library-to-library resource sharing is secondary.

Ralph: Intends to see this used as “customer” being internal OCLC … B to B internally. Don’t expect ZNG will displace Z39.50. Wants to see this adopted outside the library community. Targeted OAI community as obvious implementer who’s refused to use Z39.50 thus far.

Sebastian: Lose of functionality between SOAP and GET?

Ralph: No.

Matthew: Recommend 256 character URL limit; modern browsers probably won’t have limit; proxies might therefore use POST rather than GET if long query.

Sebastian: GET is easier.

Ray: SOAP is going to become more widely used soon.

Matthew: Assume toolkit with POST and GET means this is trivial.

Sebastian: Wants toolkit.

Ray: Cookbook or primer is coming.

Ralph and Matthew: Will be cut and paste-able.

Austin: Encorporating search and present will turn them off because of length of time; like to be able to do search and go away for 4 hours.

?: This can be done by requesting zero records.

Poul: “Time to live” parameter will ensure the result set will be kept and you can log in from another terminal to get it on another day.

Ray: People have always wanted to build present into search.

Adam and Ralph: Lots of stuff still needs to be done.

Matthew: In WSDL spec you’ll see it’s noted as prototype 1 rather than version 1 … therefore it’s still a draft.

Ray: All ZNG activities will be highlighted by Z maint agency regardless of who’s doing them.

Bill: Anyone can participate in ZNG.

Mark: I’m building a client.

Adam: Is there a timescale for version 1?

?: No. Join the prototype group because ZIG involvement will shape the development of this.

Bill: Purposely did stuff before this meeting so there’d be implementer experience to share.

Rob Sanderson: Rather than sending files, send pointers. Why not get rid of all formats and return just pointers.

Ralph: Good idea for alternative schema.

Matthew: If generating data on the fly, this will not work. DC is the only mandatory format at the moment because the others are not always relevant.

Ralph: If multiple pointers you’ll want to send some metadata to distinguish them.

Mike: Request schema encapsulates element sets and record syntaxes?

?: Yes.

Rob Bull: Why no scan?

Ralph: Scan is clearly another service and will be pursued.

Poul: Architecture makes it easy to add other services.

Ralph: Hopes for ZThes as well.

Pieter: Where are things with implementation?

Ray: Test bed planned for mid-Sept. but did not make it; timeline not defined.

Ralph: Working server within a month.

Mark: Still working on client.

Joe: What is the process from now on?

Ralph: Wants all projects to move forward.

Ray: Not ready to standardize yet.

Peiter: How can they (projects) all live together?

Joe: Concerned about everyone doing something different.

Ray: See how this pans out; ZIG may not be the deciding body on this; W3C struggling with what web services are; standardizing a web service should be relatively easy.

Mark: Will have to decide someday how to phase out the old stuff.

Rob Sanderson: How do people feel about ZNG who came to this meeting just to hear about it?

Adam: Will be moving forward with something.

Ole: Interested and thinking about it.

John Lowry: Not sure how/when to apply.

Ray: Strong vote of confidence will move us forward more quickly.

Ben Soares: Knows SOAP and worries about people taking it up before it’s out of protocol state.

Rob Bull: Use of XML becoming more prevelant.

Pieter: Worried this morning because so much was being cut out; feels better now because functionality is still there.

Lennie: Mind is more open than before.

Rob Sanderson: Likes it better now.

Bill: Cilla Caplan (NISO standards chair) hearing good things from external people that are interested.

Open URL (Thomas Place):

Note: Herbert V. cancelled at the last minute, Thomas was asked to do a presentation on very short notice, and graciously agreed.

power point slides

• user connects to information source that is OpenURL-aware (e.g. Medline), users browses or searches, user finds citation with OpenURL link to a “service component” (SFX, Names, iPORT) which component depends on the user, user activates link, service component selects extended services using metadata identifier in OpenURL
• Links to extended services include: holdings information from a catalog, full text, pre-prints, other publications of the same author, publications that cite a given publication, etc.
• OpenURL spec:
• object description: syntax for encoding a reference to a scholarly work
• origin description: elements that describe
• =base-URL? [origin-description] object-description

Question/Comments:

Ray: does this have to be an HTTP URL?

Mark: discussions that it might not but assumes so right now

Mark: private zone can contain authentication information

Ralph: header contains referring URL

Joe: are the metadata pieces standard? so isn’t this just inventing a query language

Thomas: this is not necessarily a query you execute.

• OpenURL is HTTP GET or POST request that delivers metadata or identifiers
• identifiers are used by the local service component to fetch additional metadata about the referenced work
• OpenURL and Z39.50 are complimentary: Z39.50 search and retrieval can return a record that is enriched with an OpenURL; Z39.50 can be used to fetch metadata; OpenURL contains some functionality not available via Z39.50

Question/Comment:

Lennie: what does the service component have to do when it gets an OpenURL

?: this is not well defined

Leif’s presentation:
Note: Leif was similarly asked, on short notice, to summarize the relationship between Z39.50 and OpenURL.

• ZNG URL is similar to OpenURL à lots of services do their own thing after the “?”
• ZNG and OpenURL attributes will overlap à why not use the same syntax for the same concept
• linking is special case of search with AND as the operand
• thinks OpenURL is missing some relevant attributes
• why not standardize what happens after ?

Question/Comments:

Andy: OpenURL is not a search language; it is a mechanism for encoding citations and a metadata transportation mechanism

Ralph: akin to known-item searching

Ray: maybe ZNG will use OpenURL query language if it gets standardized

Mark: OpenURL not charged to standardize anything but syntax

Ole: danger that we are confusing OpenURL syntax with Open Linking framework; not wise to merge syntax of ZNG and OpenURL

Sebastian/Mike: OpenURL more parallel to record syntax than query language

Lennie: agrees that they are different

OAI - Andy Powell powerpoint slides

Discussion: (coming)

ZOOM: The Z39.50 Object-Orientation Model -- Mike Taylor

Discussion: (coming)

End of Day 1