White Paper


An Open Media Manager For Mass Storage Systems

Sanjay Ranade
Infotech SA Inc.
Silver Spring, Maryland

December 1997


Introduction

OpenVault is an open media manager for UNIX® and Windows NT® systems being developed by Silicon Graphics, Inc. It is designed to ease the integration of many different types of robotic storage libraries with storage management systems and to provide the basis for an industry-standard media manager.

OpenVault provides a comprehensive set of media management services through a standard interface for robotic libraries, loadable drives, and client applications. These services can be used by storage management applications such as backup, archive, and HSM (hierarchical storage management).

OpenVault is the first open media manager that, through its developer, SGI, will be available to any storage management application vendor. SGI's stated policy is to actively promote and encourage the use of OpenVault not only for storage management applications running on its own platforms, but also on any other open platform. Backed by the support of a major computer vendor, OpenVault offers the prospect of the first truly portable and widely usable media manager.

OpenVault fulfills a critical need that is faced by today's storage management software application developers. Each application that requires the use of robotic storage libraries must use a separate vendor-developed device driver and additional related code for each library. There is no single standard interface and set of functions for managing and accessing different robotic libraries from different platforms. OpenVault is designed to provide such an interface and such a set of functions.

OpenVault is currently available on SGI IRIX® systems, and ports of the software for other common systems such as SunTM SolarisTM, HP/UX, Windows NT, and others are planned. OpenVault will be used by Legato NetWorker and SGI's Data Migration Facility (DMF) software for storage management applications such as network backup and HSM.

This white paper describes the functions and features of OpenVault. It provides a detailed technical description of the software and covers its architecture, interfaces, and administrative commands. The benefits and advantages of using OpenVault for new applications or to replace existing media managers are also fully explained.


SGI and Mass Storage

SGI is a major player in the high-end mass storage systems market. SGI's storage management strategy is based on providing full platform support for third-party storage management applications vendors. SGI has actively promoted the development of industry standards in mass storage through support of working groups such as the DMIG (Data Migration Interface Group). The development of OpenVault is an internal company-sponsored effort with the goal of providing a standard media manager for all open platforms.


Need for Standard Media Manager

Despite the proliferation of data, many information professionals have trouble accessing the information that they need quickly and efficiently. Secondary storage on disk drives is usually near capacity, and is generally devoted to system overhead and working files. Tertiary storage often contains the desired data, but is reachable only after expenditure of time and effort. Attentive management of removable media libraries can enhance the availability of information without significantly increasing overall system cost.

The traditional way of dealing with robotic libraries is with specialized applications that interface to particular libraries and drives. Generally, devices are monopolized by a single application.This approach has several shortcomings:

  • Manufacturers of robotic libraries and drives must develop device drivers for each new product on all supported system platforms
  • Software vendors must develop additional code to integrate new robotic libraries and drives, resulting in product support delays
  • Computer system providers have a difficult time offering a complete range of robotic libraries and applications when customers want the full range of libraries available
  • Users and administrators have no access to the removable media library except as granted by a specialized application--sharing is not possible. To solve these problems SGI has developed OpenVault, the industry's first media manager for open UNIX systems. OpenVault provides a set of standard interfaces that raise the level of abstraction, enabling rapid deployment of removable media libraries, drives, systems, and client applications.

OpenVault Overview

OpenVault is software that provides media management services required by storage management applications that store and retrieve data from robotic library devices. These media management services include the naming of media, the tracking of media by library slots, the tracking of media contents, the loading and unloading of media from drives, the import/export of media, and others.

Examples of applications that can benefit from the use of OpenVault include network backup, archive of data for long-term storage, HSM, CD-ROM library management, broadcast video tape management, and others. These applications typically require access to large amounts of data on media stored in robotic units.


Technical Description

The OpenVault software consists of a number of cooperating processes running under a host operating system. Taken together, these processes and their common databases provide a comprehensive set of media management capabilities that allow sharing of robotic libraries between different applications running on a host platform.

Architecture
OpenVault consists of the following major components:

  • MLM server -- the central component is called the Media Library Management (MLM) server. This server maintains a database of media units in the library and information about authorized applications, drives, and libraries.
  • LCP -- for each removable media library, there is a library control program (LCP). The LCP and the MLM server communicate through an interface known as ALI (abstract library interface) and ALI/R (abstract library interface response).
  • DCP -- for each drive in the library, there is a drive control program (DCP). The MLM and the DCPs communicate with each other through an interface called the ADI (abstract drive interface) and the ADI/R (abstract drive interface response).
  • Persistent storage -- a database with which the MLM server keeps track of tapes under management.
  • A command interface to administer the MLM server.

The relation of these components is illustrated in Figure 1, Overview of OpenVault Operation.



Media Library Manager Server
The purpose of the MLM server is to coordinate library management requests from client applications and, after translation, pass them to the appropriate LCP and DCP. The MLM is also responsible for creating and maintaining media pools for each application and for mapping logical media names used by applications to the physical media units in a library device.

The MLM uses the LCP and DCP for operations such as the mounting and unmounting of media units. Once mounted, the client application reads or writes the media independently of the MLM using POSIX standard I/O interfaces. Media mount and access points are illustrated in Figure 2, Media Mount and Access Points.



The MLM facilitates the addition of client applications, libraries, and drives to the system without modification of OpenVault. New programs are installed and administrative commands are run interactively on a live system to inform OpenVault of the newly added programs.

Media Unit Naming
Since different applications have their naming schemes, OpenVault allows clients to select their own names for media units. The client name spaces are kept distinct so that different clients may use the same name for the same or different media units. Media units are protected between applications but can be shared or transferred if so directed by the administrator. Bar code handling is allowed through the appropriate LCP and DCP. These programs must read the bar codes and inform the MLM server of the corresponding physical locations.

Communications Paths
Communication between the different components of OpenVault is through the Client Applications Programming Interface (CAPI), Abstract Library Interface (ALI), and Abstract Drive Interface (ADI). These interfaces are composed of ASCII strings that travel between machines across TCP sockets. If the host on one side of the connection becomes unavailable, a signal is sent to the host on the other side of that connection. This underlying communications layer is hidden in a library from the applications developer.


OpenVault Interfaces

Various OpenVault interfaces are available to the programmer. These include the CAPI, the ALI, and the ADI. These interfaces are described below.

Client Applications Programming Interface
The CAPI is an ASCII string language used by client applications to communicate with the MLM server. After submitting each command, the application waits for the server to acknowledge receipt of the command. It is not necessary for an application to wait for the completion of a command before sending the next command. Communications with different libraries can also be synchronous if necessary.

Abstract Library Interface
The library control program is the OpenVault component that performs the low-level operations to control robotic libraries. The MLM server issues directives to the LCP through a language called the ALI. The LCP replies to the MLM using ALI/R. The developer's kit includes parsers for these languages. The list of ALI commands include mount, unmount, move, eject, openport, scan, activate, reset, exit, and more.

Abstract Drive Interface
The drive control program is the OpenVault component that manages the configuration of the drives and performs the drive control tasks associated with CAPI mount and unmount requests. There is at least one DCP corresponding to each MLM managed drive. The language used by the MLM for communication with the DCP is ADI, and for response it is the ADI/R. The ADI includes commands such as attach, detach, load, unload, position, identify, and others.

A layered view of OpenVault is given in Figure 3, OpenVault Layers.




OpenVault Administrative Commands

OpenVault administrative commands are executed from the system prompt. The administrative command set includes status and configuration, entry/modification of media units, enter a media unit, eject a media unit, mount unit in a drive, move unit from one slot to another, and others.


OpenVault Availability

At the time of writing (March 1998) OpenVault Version 1.0 is available on SGI IRIX. Version 1.1 is expected to be available in the second quarter of 1998. The software was developed under IRIX 6.2 and IRIX 6.4.

A port of OpenVault to Solaris will be completed by Legato Systems and will be returned for maintenance to SGI. Legato will use the OpenVault code for the NetWorker products initially on SGI and Sun servers. Ports to other platforms will be available at a later date.

SGI's DMF (Data Migration Facility) is another application that will use OpenVault for media management functions. DMF's use of OpenVault will be available with Version 1.1 of OpenVault when this is released in the first quarter of 1998.


Industry Acceptance of OpenVault

The broad acceptance of OpenVault on the other main open platforms (RS/6000 AIX and HP/UX) depends on how quickly Legato ports and deploys OpenVault on these platforms. Other storage application vendors (particularly vendors such as EMASS and Spectra Logic that are often used with SGI platforms) are examining OpenVault and are expected to use it for media management function with their respective backup and HSM software products. It is too early to ascertain the position of other storage management vendors such as Storage Tek, IBM, and Veritas.

To help promote OpenVault, SGI has opened an ongoing dialogue with storage library vendors to take the steps necessary for supporting a large number of library devices under OpenVault. SGI has also undertaken to give the OpenVault source code to key customers writing custom storage management applications. In working with IEEE standards groups, SGI will supply OpenVault specifications that the standards working group will modify and then issue as the official IEEE media manager standard.


Summary

  • OpenVault is the first media manager developed for open UNIX systems. Its initial implementation is available today under SGI IRIX 6.2 and IRIX 6.4. The software is being ported to other UNIX platforms
  • OpenVault fulfills a critical need faced by today's storage management applications developers. Instead of having to use vendor-supplied device-specific code for different library devices, applications developers can take advantage of a single standard library interface for different libraries on different machines
  • The library units currently supported by OpenVault under IRIX 6.2 and IRIX 6.4 include the StorageTek 97xx, EMASS AML and the IBM® 3494 libraries
  • The applications that currently use or will shortly use OpenVault include Legato NetWorker and SGI's DMF. Other applications (such as Spectra Logic and EMASS software) are expected to use OpenVault in the near future
  • Storage management applications vendors such as Legato and possibly EMASS and Spectra Logic may standardize with OpenVault on other platforms. As acceptance of OpenVault grows, other application vendors are expected to follow suit. The IEEE standardization effort, using OpenVault as a kind of "baseline," will increase its acceptance in the industry

Bottom Line

OpenVault offers the first real prospect of an industry standard media manager for open platforms. SGI participation in the IEEE standards group and its use of OpenVault as a baseline should ensure that OpenVault is rapidly adopted by the major storage management application vendors.


About SGI Mass Storage Solutions

SGI targets the network mass storage systems market with a broad range of third-party storage management applications. Legato Systems' NetWorker, the preferred network backup and recovery solution, is actively supported by internal staff and marketed by the sales forces. Solutions for hierarchical storage management include the Data Migration Facility marketed by SGI, AMASS Data Manager marketed by EMASS Inc., and the UniTree Hierarchical Storage Management System developed and marketed by UniTree Software Inc. High-speed SGI platforms such as CHALLENGE® and the Origin® 2000 provide the very high I/O capabilities essential to the data movement workload that is characteristic of networked mass storage systems. Mass storage systems driven by SGI servers are in use at a number of large data management centers such as those at Sandia National Labs, NASA/Goddard Spaceflight Center, and many others.


About Infotech SA Inc.

Infotech SA Inc. ( infotech@access.digex.net ) was founded in November 1990 to research mass storage markets, technologies, and systems. The company publishes a detailed annual report on these subjects for help in making informed purchasing decisions. The first Mass Storage Report was published in 1991 and marketed by "word of mouth" to organizations involved in the purchase, development, or use of mass storage systems. In 1998, its eighth year of publication, the Mass Storage Report is purchased by organizations worldwide.

To order or for more information, call Infotech at (301) 890-2114.


Copyright 1998 Silicon Graphics, Inc. All rights reserved. Specifications subject to change without notice.
SGI, IRIX, CHALLENGE, and the SGI logo are registered trademarks, and OvenVault and Origin2000 are trademarks, of Silicon Graphics, Inc.
UNIX is a registered trademark in the U.S. and other countries, licensed exclusively through X/Open Company Limited.
Windows and Windows NT are registered trademarks of Microsoft Corporation.
Sun and Solaris are trademarks of Sun Microsystems, Inc.
IBM is a registered trademark of International Business Machines.
All other trademarks mentioned herein are the property of their respective owners.