ZIG Meeting

  Opening Plenary Meeting

Notes provided primarily by Les Wibberley.

Report revised 2/8/01

• Reviewed agenda for the week...
• Register of implementors (Ray)

Introductions

• Sebastian Hammer, Index Data, Denmark; consulting projects on Z39.50; maintain several freeware packages
• Fretwell Downing - client side Z39.50 to create ill
• Elliot Christian - GILS
• Oxford University - Z client and server; WAP to Z39.50 gateway
• University of Leuven, proposals to build virtual union catalog; Elias system; digital library system via Z39.50
• Joe Zeeman - Z39.50 for corporate Intranets and enterprise portals, wrapping z into EJBs for J2EE environment
• U. Kansas - natural history data, using Z39.50
• British Library - amiss system, Z39.50, develop new external Z39.50 access to their databases, academic access to internal db
• Fretwell Downing -  virtual union catalog
• Mike Taylor - TECC - collaborating with index data - Perl module for building Z39.50 clients;  european project to build federated multilingual search system, using plug compatible brokers
• Capitol District Library - library consortia in Albany NY, statewide product to connect libraries using Z
• Margery Tibbetts - California Digital Library - OPAC, Z39.50 in SearchLite for students, digital library, open archives
• Ralph LeVan - OCLC- in SiteSearch group, using Z39.50, used to support FirstSearch product also, CORC service also uses Z39.50 - online internet cataloging; online Z39.50 cataloging access to WorldCat database;
• Jacob Hallen - Sweden Royal Library - updating Z39.50 server, ONE-2 project, building national virtual union library
• Danish National Library   lib - test project as part of danish library, will launch web gateway to library
• Danish Biblio. Center - operates danish national union catalog, library applications, using Z39.50 for all production systems, international standards, including ONE-2.
• Larry Dixson - Library of Congress - use Z39.50 servers from Endeavor (Voyager, etc.)
• Slavko Manojlovich -- Memorial University of  Newfoundland - supporting Bath and texas profiles, doing interoperability testing, server available soon
• Barb Shuh - National Library Canada - ILL tech. editor; working on bib-2 attribute set; Z39.50 clients and servers to NLC and virtual library of canada
• Lennie Stovel - RLG - released ILL manager which uses Z39.50; working on OPAC implementation, working on holdings schema
• Janet Gatenby - PICA - involved in holdings schema and attribute and union catalog
• epicstech - variety of Z39.50 clients and servers many compliant with Bath
• Bob Waldstein - Lucent - Z39.50 servers running over 5 years; use to access external resources, go to ProQuest for images and articles
• Mark Needleman -  DRA; variety of Z39.50 clients and servers across several product lines; upgrading from v2 to v3 now; NISO liaison for new profile
• Ray Denenberg - LC, Z39.50 Maintenance Agency
• Ralph Orlik - LC

Current Status  (Ray Denenberg)

• NISO needs to reaffirm its standards every 5 years
• Z39.50 is now up for reaffirmation/review/comment
• discussing some fairly substantial changes to Z39.50 at this meeting - this is separate from reaffirmation
• time to merge approved ZIG changes/clarifications, etc., into the standard itself.
• this might mean including these changes in the reaffirmation process
• OSI lingo in the standard - clean up antiquated language
• more than reaffirmation, less than issuing a new version of the standard -- a "revision"
• prepare draft ready for ballot addressing these issues
• on Friday, Ray will discuss the status of the revision; not ready for review yet
• timeline for the revision
• probably by June will have a draft ready for review
• 63 clarifications that have to be worked into the standard
• There are certain objects and items which will be removed from the standard, based on the last meeting
• Ray will provide a pointer to the document describing what will be removed or changed (or hardcopy)
• this was discussed in conjunctions with NISO SDC, and discussed at Leuven meeting
• Relationship between this and the ISO standard?
• ISO would need to decide what it will do with its standard, within the context of ISO TC46
• Sally McCallum is still chair of ISO TC46/SC4/WG4
• Concern about NISO/ANSI version standard possibly diverging from the ISO standard
• ISO standard was approved at a later time than the NISO standard
• Intent to follow NISO approval with ISO approval process of the revision
• The ZIG is really the advisory group for the ISO standard

ZIG Survey Results

(presentation) - have hardcopy of the slides

Note: not all ZIG members received the survey

Q: what was the purpose of the survey: for NISO, Maintenance agency and ZIG

to get some input for future directions

NISO board will discuss future of Z39.50 in mid December

received about 22 responses

Reviewed responses

some of the problems are perception problems

lack of standards for indexes is one issue

encoding schemes

documentation not clear, too formal, too much OSI terminology, difficult to read

ASN.1 vs XML

comments on this included in survey

issue of replacing ASN.1/BER with XML encoding

Version issue

variety of yes/no/maybe answers

Administrative questions also asked

no statistical/ numerical analysis performed yet

would be interesting to know who Responses came from (role, etc.)

May want to do a follow-on survey

What do we want NISO to do with the results of this survey

Issue: with a small number of respondents, limited to a small subset of people (not ZIG members)

 is this a valid cross-section of opinions, etc.

this survey wasn't required by anyone

Ray and others put together some of the questions

several of the comments are in fact valid, and should be addressed

misperceptions included:  - many of these are implementation issues

diagnostics displayed to user vs protocol diagnostics

replacing ASN.1 with XML would not simplify the standard

display of MARC records is not a protocol issue

some valid feedback about implementations, that are not actually problems with the protocol or the standard

we need to discuss the ASN.1/BER vs XML issue further in another session

how about Z39.50 over HTTP or other transport systems

issue of running directly over TCP/IP and having to address all the problems that come up

need to address issue of using Z39.50 in modern development environments like EJB/J2EE

there were some perceived problems with the survey by the ZIG

do we want to spend more time and resources on this survey, or should we ask more precise questions

implementation time frames were not specified

perhaps do straw poll here at the ZIG

perhaps do some telephone interviews with registered implementors, etc.

this is independent of NISO reaffirmation of the Z39.50 standard

this addresses future directions for Z39.50

Provocative Presentations

A.  Future Proofing Z39.50

• Speaker: John Kunze, UCSF and NLM
• Z39.50 is only standard that addresses internet searching
• Z39.50 is still encumbered with some remnants of OSI
• emergence of lean protocols and bulky markup (SGML)
• HTML adopted by the web, disposed of DTD requirement
• difficulty of getting GRS-1 adopted as record syntax
• Z39.50 group dynamics
• collective < sum of parts?
• how to leverage past knowledge and experience
• how to keep evolving the standard
• long-winded discussions on less important topics
• debate issues endlessly rather than letting someone else take care of these non central issues
• problem not unique to ZIG: similar problems in Dublin Core and IETF URI working groups
• the rise of worse-is-better
• the right way - MIT school way of designing - lisp language
• the worse is better way - unix/c
• inconsistency, lack of simplicity
• better chances of survival
• right way gives endless design process and difficult implementation
• Z39.50 design
• OSI terminology still haunts us - scares people away
• use simpler terminology - unlike PDU, RPN, etc.
• ASN.1 is for Albatross, BER is for Bummer
• this is a major hurdle to overcome
• from the "OSI right design" school
• string based encoding is a much simpler, common convention, printable strings
• HTTP, email, etc. all use these simpler conventions
• dealing with library-centric reputation
• focus on MARC records
• try to define what problem Z is trying to solve
• implementation burden of MARC
• similar burden to BER
• comparison with internet search engines
• what Z39.50 features could we live without?
• result set management not sacred
• statefulness not sacred
• cram most stuff into the urls
• piggybacked search
• default mode of HTTP search engines
• scan, sort, explain
• can be expressed in search
• internet style workaround possible
• INIT, close and A&I control
• state can be passed in cookies, etc.
• Z39.50 is at a crossroads
• approach from a new direction - clean slate
• IR over HTTP
• e.g., Google search query
• look at url returned - includes search terms buried as parameters
• example of ad hoc peer-to-peer search protocol
• example of present request
• NLM Entrez interface
• www.ncbi.nlm.nih.gov/entrez
• ASN.1 encode records
• entrez-Z39.50 gateway underway
• XML factor
• slightly cleaned up version of SGML
• designed for text, but still has some problems with text
• division between format and semantics is still and issue
• DTD is an issue - why have it?
• not designed for metadata or protocol control information
• but it has pull, and we need to deal with it
• has even more "too many options" than Z39.50
• alternative to XML:
• what works on the internet?
• rfc 822 headers:  label: value syntax
• but this is not good for nesting records
• Slash and burn, start fresh
• salvation through suffering
• Discussion
• What is the impact of the XML Query work
• far from done
• no syntax defined yet
• no world changing stuff really coming out of W3C
• An awful lot of effort is going into XML
• importance of being able to structure records
• database records are very rich now
• Q: plan for holding records

B.  Paul Henrick Jorgensen

• topic: Z39.50 Issues
• background
• take-up within selected communities is growing (adoption)
• growing number of web-Z gateways - this works very well
• constrained to metadata resources - not used much to handle multi-media
• used for parallel distributed searching for a number of the european projects, even though excluded from the standard
• few advanced services implementations - most are simpler implementations
• compatible international profiles -
• Issues
• confusing specifications to outsiders
• lots of baggage carried along
• could be cut down significantly, could use much simpler terminology
• Z39.50 is completely ignored by mainstream systems and vendors
• even  some library systems vendors only pay lip service to the requirement
• few toolkits available, and all are proprietary
• interoperability is problematic
• even with same protocol and profile, there are interoperability problems
• limited applications logic
• relative modest: search and retrieval
• no object oriented facilities
• most new college grads expect OO tools
• closed ZIG fraternity
• Objectives
• prune obscure facilities and OSI baggage
• specify streamlined Object RPC API
• align with current technology
• encourage support by major vendors
• focus on applications framework - that is what is happening
• popularize ZIG activities
• more open atmosphere at meetings
• Solutions
• review of version 4 features
• prune obscure facilities
• Specification of Z39.50 over SOAP
• note this is currently being successfully done
• technology, tools, Object AP
• lots of tools are becoming available for XML environment
• Bath profile development
• mainstream support
• New ZIG format
• open forum for proposals and tutorials
• significant improvement over the past
• Application Focus
• not clear how to address this
• Summary
• Z39.50 persists, mainly within libraries
• basic interoperability is achievable
• could be run over other transports
• Technology needs to be modernized
• obscure features should be dropped
• some of the extended services (extended services database)
• espec-q - will anyone ever implement this?
• unclear relationship to ILL and Circulation
• new ZIG format and procedures
• We should take a lesson from the EDI/Edifact community and efforts

C.  Matthew Dovey

• Provocative points
• standard difficult to understand and alien to internet developers used to RFC type descriptions
• lack of tools
• ZIG seems difficult to approach/work with
• ZIG seems library oriented not IR oriented
• ZIG needs to market Z39.50 better
• Z39.50 is not heavy weight or difficult, buts gives that impression
• world is not going to beat a path to the door of searching and IR experts, but will go their on way
• Z39.50 should be a component working alongside other standards, not reinvent them
• e.g., extended services
• Z39.50 should be less visible (e.g., web developers rarely know how the HTTP protocol works) however, poor Z39.50 implementations display diags. to end users
• Z39.50 failed to provide solution to interoperable search (flexibility of standard, lack of profiles, etc.
• Z39.50-2005 as a protocol template - possible future direction
• communities have their pet protocol layers (CORBA, RPC, SOAP, XML/HTTP, ASN.1/BER)
• far easier to gateway between these if the information packets are semantically similar
• lever Z39.50 as a transport spec, protocol a profiling issue
• IDL is another example of a protocol template

D.  Mike Taylor

See Mike's report "Z39.50 - The Empire Strikes Back" at http://www.miketaylor.org.uk/tech/esb.html

Next meeting

• perhaps in about 9 months in UK - September 2001
• what should future meetings be like
• tutorials, topics, where it goes in the future
• do we want to radically redefine the standard?
• how do we get more people here?
• redefine scope as network information retrieval in general?
• how do we market the content of the meetings
• Who is the ZIG - the community that makes this decision to make radical changes
• expand the scope to information retrieval
• can we get the level of participation to define another version of the standard
• URL session - scheduled for Friday morning; reschedule for Thursday?
• John Kunze will not be here Friday
• Ray: no, can't swap that session with Thursday
• XML query and XML general discussions are locked in together
• Application oriented sessions would drive the url and explain sessions
• hard to reschedule the other Thursday sessions
• Perhaps schedule a ZIG meeting in conjunction with other types of meetings
• e.g., IETF, W3C, SIG-IR, etc.