Wednesday, January 21, 1998
Orlando, Florida
Status reports were posted separately.
Dekkers requested that the index for the registry on the web be rebuilt. Denenberg will do it.
At the EFILA meeting in November, the major agenda item was the work being done in Europe on Z39.50 profiles. The MODELS profile was presented over a year ago; the idea is being progressed through several projects. There is some interest in defining a European profile under the auspices of the consortium of European national libraries, and initiatives in Italy and France to link libraries within these countries via Z39.50. These profiles were presented at the EFILA meeting. There were big similarities between the profiles (e.g., UNIMARC or USMARC format records), but some differences. The attribute lists usually contain the ATS attributes plus national conventions. The group recognized that many people are doing the same things and arriving at the same conclusions, so they decided that Europe will prepare a European profile. The profile was prepared and is now being discussed, with voting for use by European national libraries as the basis for all national work in all countries in Europe.
The new European CNL profile is based on the Finish profile, ATS-1 and the ONE project. It defines eight Use attributes and several Structure attributes (total of 11 attributes). The profile contains requirements for servers and clients. For example, the origin must support at least one Author, Title, Subject, etc.; the target must support all eleven attributes. What does "support" mean in this context? It must mean more than just recognizing an attribute and not breaking. Dekkers' opinion is that implementors should support what is in the profile if they say they use the profile. On the issue of record syntaxes, the profile mandates USMARC and UNIMARC: the client must support one; the target must support both. Character set negotiation is not yet mandatory, but if it is done, it must be done in the way specified in the profile. The profile mandates the use of the Init service authentication parameter. It is expected that some of the things in the profile will be further developed and other things mandated. EFILA sees this as one of their important topics for 1998. Europe considers the profile issue to be very important. They also perceive various levels of profiles. ATS-1 is a baseline profile for all of Europe; other profiles will build on top of that by specifying additional features. For example, an international profile is built on top of the ATS-1 base,then a national profile is built on top of the international profile. Another point in the discussion: when you access a server, how do you know what profile that server is supporting? How can this information be communicated among clients and servers? Voting is underway on the European profile by the consortium of European national libraries. Voting should be finished and the profile established in two months.
Turner: how do we coordinate the North American profile with the European profile? Dekkers: the European profile is more or less a bibliographic profile, but it would be good to harmonize this world-wide, rather than having it be restricted by geography. Look at the European profile (release 1) on the web; we can discuss this after May 1998. Dekkers is uncertain how long the turn-around time will be on revisions to the European profile -- hence, Turner should look at it now so that they can harmonize things before people begin implementing the profile in Europe. The North American profile is a subset of the European profile (e.g., does not include Scan). P.Henrik suggested harmonizing the common elements now and addressing contentious issues later; he is optimistic that turn-around time on revisions will not be too long. He will post the URL for the Danish profile to the ZIG list for review. Turner should look at the European and Danish profiles, and maybe attend the next EFILA meeting to present the North American profile. Denmark is looking at ItemOrder and holdings now.
The profile has not changed much since Davidson's presentation at the Washington D.C. ZIG in June 1997. They were waiting for a constituency. Now there are nine constituents in the UK: national libraries and what he called "hybrid" libraries (multimedia, distributed). Several projects are underway, including four union catalogs:
The OPAC SBN in Italy is a national library service and network that has done shared cataloging (a Z39.50 centralized union catalog) and ILL since 1992. They now have about 700 libraries (public, private, academic) involved on the SBN central system, which is financed by the Ministry of Culture. The project has implemented a free, easy to use catalog with more than 4 million items hosted by the English SBN. There is also a national union catalog with 3 million titles (from 1831 to present). Other databases cover earlier works and different kinds of items (e.g., music, ecclesiastical). Graphical (Windows) and character-based user-friendly interfaces are available, and there is a Z39.50-web gateway. UNIMARC was selected (but not yet implemented?) for the data format. The main features of the SBN profile are Z39.50-1995 (v2 and v3), including Init, Search, Bib-1 attribute set with extensions, Present, Scan, Sort. Future work includes Access Control, and Extended Services (ItemOrder) integrated with SBN ILL. They also plan to do shared cataloging, a union catalog, and Explain. Significant contributions were made by some universities. The Z39.50 clients were developed by libraries participating in the project.
Supporting NASA on the development of CEOS and CIP prototypes. CEOS (Committee on Earth Observation Satellites) members are space agencies involved in remote sensing projects and brought together to develop interoperability. The goal of the committee is the harmonization of activities -- including content, format, description -- and agreement on systems and services. The Protocol Task Team (PTT) developed the design, user requirements and Cataloging Interoperability Protocol (CIP) specification for connecting the data catalogs of CEOS member agencies, and are implementing prototypes to evaluate CIP concepts. CIP is a second generation protocol based on Z39.50 version 3. They are working on an Interoperable Catalog System (ICS) for CEOS member agencies. Attribute sets and semantic definitions are derived from input from various groups. They are currently broadening the scope of the CIP profile to use Z39.50 services across domains (e.g., Bib-1, GILS, GEO, CIP databases), and trying to align the GEO and CIP profiles with minimal changes to both (attributes, architecture, presentation of results, communication, etc.) to enable CIP/GEO interoperability via Z39.50. The key components of CIP (v3) space are Clients which connect with Retrieval Managers, which then access the GEO servers via Translators. GEO (v2) space consists of Clients, Gateways, and Servers. Problems that had to be solved include: element mapping, compound attributes (not a good idea), interoperable minimum Use attributes (based on GEO minimum set), and element set alignment (defining a common set).
Issues for the ZIG: has anyone done a full implementation of Explain or a full implementation of Persistent Result Sets and Persistent Queries? Waldstein did Explain. Graubert (NOTIS) was working on Explain but stopped six months ago after implementing most of the records because most of what they needed to configure their clients they could not find on servers (e.g., servers say they support Bib-1, but don't specify the attributes). Wibberley: not having Persistent Result Sets certainly limits the functionality that people want, but they haven't done it yet. Hinnebusch: perhaps the reason people haven't done this is because most of the ZIG is libraries and they are not the primary customers for Persistent Queries. Tibbetts: we have the concept of Persistent Queries (or query schedule that returns only items that are new since the previous query), but not Persistent Result Sets. There was some discussion of whether this is a client- or server-side function. Denenberg: Persistent Queries and Periodic Query Schedule are two different Extended Services; the schedule references a persistent query, but not vice versa. The ZIG agreed that special libraries (e.g., medical) and agencies are more interested in persistent queries than academic and public libraries.
Future CEOS/CIP work: development of prototype Translators, release A, preparation of release B, ....
See also http://www.ceo.org/ and http://ceos.ccrs.nrn
They plan to release the "final" version of the profile next week. The new version includes a few changes, expansions and clarifications. For example, they considerably expanded the conformance statement and the discussion of updating or replacing an item. The profile has been implemented. Gatenby's company, Stowe, is testing version 3. The client is available commercially; an evaluation copy is free via FTP. (Update can be done using Z39.50 or FTP?) A working draft of data elements 84596 is available in softcopy (from an ISO subcommittee of TC46) based on Z39.50 and the union catalog profile. The National Library of Australia signed a contract with AMICUS last week, which includes implementation of the union catalog profile on the server side.
The CIMI interoperability testbed -- including twelve participants, three clients, eight servers, and multiple databases -- finished successfully in October 1997. Participants included Blue Angel, Crossnet, Finsiel, Joanneum Research, System Simulation. The testbed was guided by an evolving implementors agreement that was a subset of the draft CIMI profile from 1996. The testbed focused on object records, images and bibliographic records using Z39.50 and HTTP. CIMI contracted with Blue Angel Technologies to build a Java client; (?) and Needleman of Univ of CA also built a client. Testbed clients and servers interoperated successfully at a technical level; the testbed did not focus on semantic interoperability. The CIMI profile can handle multiple images (renditions) associated with a record and can provide metadata about each rendition. The twelve participants did a lot of work in a short period of time.
Testbed participants tested the Z39.50 specifications, Java client (Blue Angel), CIMI server toolkit (System Simulation Ltd), and clients and servers that support the CIMI profile. The next steps are to revise the profile based on the testbed experience and to provide implementation guidance and mapping guidance through crosswalks. The revised profile will be release 1.0 of the profile, and will be made available on the web. They also plan to progress the CIMI profile through the IRP process, after aligning it with Aquarelle. Future work entails looking at cross-domain searching, navigation, a metadata project to test implicit assumptions in Dublin Core, and work on identifying levels of conformance for the CIMI profile.
Hinnebusch: any plans by testbed participants to move CIMI into production? Selway: yes, projects in UK and Aquarelle are interested in putting CIMI into production. Moen: CIMI is being referenced in RFPs. Client developers are looking at this as well. The four levels of conformance to CIMI are included in the profile (that section to be revised and made available on the web).
There has been very little activity from OMB toward implementing GILS. There is no regulatory guidance, but at a technical level, the profile is ready.
See also http://www.cimi.org
What has happened to the Explain testbed group started by Dekkers "in a previous life two years ago"? Can Bull tell us about the status?
Bull: the Explain testbed discussion group has been very quiet. Denis Lynch was going to revisit the idea of putting Explain on a SilverPlatter server; it was there once and quite good, but it disappeared months ago. From the ONE project, Explain was one of the main criteria for providing user assistance and client configuration information. There are seven targets that offer Explain service in different levels of implementation and two Explain subsystems (Joanneum and Crossnet). Crossnet produced an add-on module for ZSQL Explain. The ONE project deliverables included guidelines on Explain; they will continue to use Explain and further develop it. CIP considers Explain extremely important, as does the UK. From the client point of view, only the Java and Windows client (developed in the ONE project) have Explain fully implemented. Several other European projects have also implemented Explain.
There are real benefits of the Explain service for clients, but keeping an Explain database up to date is tedious. It would be nice to have a profile OID added to Explain, and it would be helpful to find information about other targets as well at one Explain server (i.e., a surrogate/proxy Explain server for servers that don't have Explain -- something like the WAIS directory services concept).
Denenberg will add OID for profiles to the ZIG agenda.
Wibberley: if this doc becomes a definitive list, will implementors have to code to it? Denenberg: where do we see this doc going? We haven't gone anywhere with the idea of structured diagnostics. Though we once thought that we would develop different diagnostic sets, we stopped with the Bib-1 diagnostic set. Levan is happy with Gatenby's doc as commentary, and sees no need to progress it beyond that or to make it part of the standard itself. Wibberley: a Z39.50 toolkit or documented API -- you would expect error return codes from an API to be well documented so that you can handle them predictably. Currently our diagnostics are just a list of error codes, not related to particular operations, which makes it hard to program robustly for different servers. He thinks it would facilitate interoperability if we documented which diagnostics are valid for which operations. Levan doesn't think this would help or change writing code. Denenberg thinks Wibberley's comment is legitimate. Levan thinks the logical extension of Wibberley's approach is to make diagnostics outside of this list non-conformant.... Hinnebusch: let's decide at the next ZIG meeting what's to become of this doc. Hinnebusch and Stovel agree with Levan: we don't need this doc as part of the standard, though it is useful for implementors to figure out what to do.
Wibberley: this discussion now relates to our age-old discussions of what goes into the standard and what does not go into the standard. He proposed that we polish this doc, maintain it on the Z39.50 Maintenance Agency web page, and title it "guidelines for diagnostics." Don't make it part of the standard but make it available. Follett supports the idea of clarifying usage of the diagnostics; they plan to mass-produce Z39.50 services, want the best interoperability possible, and would find a classified list of diagnostics helpful. Gupte: it is impossible to have a perfect set of diagnostics, but anything beyond what we have now would be helpful.
Then there was some discussion about the importance of interoperability, which may be facilitated by this list of diagnostics. Denenberg divided the discussion into two threads: (1) the need for a classified list of diagnostics (which may attempt to be comprehensive or only a subset of everything possible), and (2) the need for a more detailed discussion of the diagnostics that would facilitate interoperability.
D Lynch: we're confusing two things. No matter what we do with documenting diagnostics, clients are going to get error messages that they've never seen before and show them to the user. What we can do is document the diagnostics that can be frequently encountered under normal operations -- so that we all agree on that. And, there are not many things that the client can actually do anything about, but perhaps we should list them somewhere along with recommendations or suggestions for what to do in these circumstances. Turner: from the user's point of view it would be nice to have such a list of diagnostics that clients can do something about, and suggestions for what user's can do. Waldstein does a lot of capturing diagnostics and trying to convert them into intelligible messages or suggestions for users. He is not sure that the wording (handling) will work across servers; it may be server dependent.
Wibberley articulated three uses for diagnostics: (1) allowing the client to dynamically respond to a situation and provide a useful response to users, (2) diagnostics for users, and (3) having granular error codes come back that enable you to figure out an interoperable situation. These would be useful in the development stage, not just production level. We really need an enumerated list of diagnostics to insure semantic interoperability. Denenberg is willing to expand the current diagnostic list to contain more semantic information and indicate the operations where they can occur.
Hinnebusch tried to summarize: the consensus appears to be that a classified list of diagnostics with additional explanatory prose, maintained by the Maintenance Agency, will be useful. Stovel's basic requirement is a reliable, numeric-ordered list of diagnostics; anything beyond that is gravy.
C Lynch: this list of proposed diagnostics raises some useful questions about why you want diagnostics. Two reasons: (1) to give the client information to take some action, (2) to pass information to the user about limitations in the client, etc. Some of the proposed diagnostics accomplish neither of these purposes. He wonders about "out of memory," "a configuration file is missing," etc. Why do we want those messages instead of something more generic? The client can't do anything about this. What are the principles for when it makes sense to have a diagnostic and when it doesn't make sense? What is the ZIG philosophy of diagnostics?
M Taylor proposed some of the diagnostics because he couldn't figure out which standard one to use, e.g., temporary or permanent system error. Maybe new explanatory prose for diagnostics will help sort this out. Perhaps we should return information abut which file is missing or send back qualifying information in AddInfo. Some of these problems should be reported to the server administrator to address a system configuration problem. (e.g., Out of memory). Several of these errors are the type of information which should be logged to the server error log file. Denenberg pointed out that the ZIG previously agreed that SQL error would be a single diagnostic, and the specific error would be carried in AddInfo. Hinnebusch suggested putting some of the non-general diagnostics in a private diagnostic set.
Then came a lengthy discussion of how to settle the issue of pending diagnostics for Bib-1. Should we vote on each pending diagnostic? No. Hinnebusch suggests that we not accept any of the pending diagnostics without accompanying semantics. Drop 1026, 1027, 1028 1033 and 1038; keep 1025, 1029 and 1037. Change 1034 to service not supported for this database. Private diagnostics should use private number space. Will reconsider those dropped if the people who proposed them justify them.
Denenberg asked the ZIG which diagnostics are to be added (whether or not they're on the list of pending diagnostics). The group responded:
There was a question about reserving 1040 to 1190 for union catalog profile diagnostics; Gatenby has already used most of these numbers. Denenberg: some of these are not really diagnostics but feedback on successful events rather than unsuccessful events. (Hinnebusch observed that a diagnosis of good health is fine in the medical field.) Stovel is concerned about mixing Extended Services diagnostics with basic protocol diagnostics. Gatenby: they relate to Update, Lock, De-dup, Authority Control, etc. In Copenhagen we decided to incorporate these into the main diagnostics and to give them their own unique numbers; her company is not happy (spending $$) changing things after every ZIG meeting because the ZIG changes its mind.
Question: are we reserving the number space for the union catalog profile and later accepting the diagnostics, or is this one step (reserve and approve)?
Graubert is concerned about the number space. Why are these numbers in the 1000s when the rest of Bib-1 is numbered much lower. Denenberg: we originally had a list of 33 diagnostics in SR; in version 3 standard we went up to 243. Somewhere, somehow we designated that new ones should start at 1000. (You can add new diagnostics, but you can't add new status codes to the standard unless you ballot.)
Stovel suggests that Gatenby's list of diagnostics be put on the web for review, rather than approve them in the abstract. (Even though they've been available in the profile for us to read since the Brussels ZIG in October 1996.) Waldstein thinks the Bib-1 diagnostic set is getting too big. He proposes an OID to designate which diagnostic set, in this case the diagnostic set for Extended Services -- though there are some already in Bib-1 for the general service (218 through 226) that should remain in Bib-1. Perhaps diagnostics for specific Extended Services should be assigned an OID.
Some disagreement about whether Gatenby should renumber to get rid of gaps in the number space. Some disagreement about whether each Extended Service should have its own diagnostic OID (since there is an OID for each service). Some disagreement of where the new diagnostic set OID should be placed: in the diagnostics tree or in the Extended Services subtree for the appropriate service (which is where we put the ES attribute sets). Denenberg wants them in the ES subtree; Wibberley wants them in the diagnostics tree; D Lynch doesn't see why it matters. Complications with private extended services (Wibberley has about 100). D Lynch: this is a maintenance issue; let the Maintenance Agency do what it wants.
The ZIG agreed to have one OID diagnostic set for each Extended Service on whichever tree Denenberg wants. They also agreed that we would follow the same procedures for assigning ES diagnostic OIDs as we do with other OIDs.
Hinnebusch will convert the Gatenby Update diagnostics to HTML and make them available on the web next week for the ZIG to review and approve.
Stovel asked a question about Digital Object Identifier. In use, is it implemented as a URL or (supposedly) a URN? Denenberg: are these all internationalString? Hammer: most are internationalString. Denenberg: we need to know the datatype to accept attributes. Hinnebusch: what is 1091 ("common code for international standard numbers as ISBN, ISSN, ISMN, ISRC, ISRN"). Hammer sacrifices 1091; all others accepted.
Dekkers: question about 1096, original language of item; don't we already have one? No, the one we have is the language of the item, which could be a translation. Shouldn't 1096 specify which code set? That's handled somewhere else somehow? Selway: 1085 through 1088 are close to saying "this is where I got this from." CIMI has already profiled this. We need to discuss this at some point.... Attributes 1085-1090 illustrate the need for the authority attribute for the next generation attribute set.