LEARNING TO USE THE VIRTUAL SOUND SERVER

AN INTRODUCTION TO VSS 3.1

Table of Contents:

1. FAQ: What is VSS?
2. Architecture and signal flow
   1. Functionality
   2. GUI
3. Running VSS
   1. Linking clients to VSS
   2. The basic client-server connection
   3. Usage
4. Sending messages to VSS
   1. Actor message syntax
   2. Usage
   3. Actor handles
5. Actor classes
6. Sound authoring
7. Audfiles
   1. Using .aud files
   2. Client-side view of .aud files
   3. Connecting clients to .aud files
   4. .aud file command syntax
   5. The simplest .aud files have no message groups
   6. Designating time in .aud files
8. Message groups
   1. Structuring client code for AUDupdate()
   2. Using message groups
   3. Assigning data from the client to Actor messages
9. CAVE Apps
10. Tutorials
11. Sound file libraries

Q. Where can I get VSS?
A. At NCSA, run VSS directly from the directory /afs/ncsa/packages/vss/6.2. Elsewhere, get it from the NCSA Audio Group website.

Q. When do I run VSS?
A. VSS runs in the background as a continuous service-provider while client applications send control messages.

Q. Does VSS work like a network or multimedia server?
A. VSS operates as a "local server" to render sound at a particular workstation. VSS can respond to multiple requests from multiple clients. Client applications can be running on the VSS host platform, or on other platforms that are networked to the VSS host. Messages are sent remotely over the network using the Universal Datagram Protocol, or UDP. (The more well-known TCP/IP is built on top of UDP.)

Q. How many VSS jobs can I run at one time?
A. Due to hardware restrictions, at any time only one copy of VSS can run on a single SGI workstation.

Q. Does every VSS client require its own VSS?
A. No. A single VSS can service requests from multiple clients simultaneously.

Q. Can I network and synchronize multiple VSS services?
A. Yes. A client may be configured to send messages to multiple servers running on separate workstations. Synchronous client requests will produce synchronous sound events.

Q. How do I make sound with VSS?
A. For making sound...

1. identify one or more synthesis algorithms in the sound server library;
2. configure a C or C++ application (a client) to send messages to VSS;

In the simplest configuration a client can send messages to the sound library as C++ function calls. In most cases a text file format -- called an .aud file -- is used to facilitate communications between the client application and the server. This tutorial introduces .aud file syntax and usage.

Q. Why use an external file instead of putting the messages into client code?
A. For flexibility and quick addition of sounds to an application, (1) the .aud file syntax provides a structured approach to making sounds, and (2) ASCII configuration files encode the client--server relationships.

Q. How do I control VSS?
A. VSS provides a library of function calls we refer to as Actor messages. Actor messages invoke the functions in Actors and are used to pass control variables to those functions.

Q. What is an Actor?
A. An Actor is a C++ class representing a transfer function or a temporal function that sends data to sound synthesis functions. Control data may be input to the Actor or it may originate in the Actor. An Actor performs a high level of sound control; for a given Actor there are usually a greater number of control signals passed out of an Actor to VSS, than the number of data streams input from a client. Actors can be used to control other Actors or to control synthesis functions external to VSS.

Q. Where do I learn Actor messages?
A. Messages associated with each Actor in the sound server are listed in the Reference manual.

The vss3.1 architecture consists of

1. a client-side library,
2. an authoring protocol stored as a text file (".aud file"), and
3. the executable sound server which has sound production engines defined as dynamically shared objects, or DSO’s

Functionality

1. Compile time: link to sound server library.
   
2. Run time: make sound requests and export numerical data corresponding to client states.

Authoring file

1. Register specific sound production messages for each type of client sound request.
2. Map domain of client-side data or status to range of control parameters for sound production.

Server

1. Parse message requests from client applications.
2. Execute sound production messages, processing data sent by client according to audfile specifications.

The goal of this architecture is to minimize the sound production knowledge encoded in the client application, and isolate specific sound production configurations in the .aud files, supported by sound engines loaded into the server.

GUI

VSS defaults to a sampling rate of 22 kHz and one channel (mono) playback. These may be changed on the control panel when it is running, or on the command line at startup. To learn these and other options see Running VSS in the reference manual.

Linking clients to VSS

Sound library functions are linked to the client application at compile time. Functions are prototyped in vssclient.h and the routine library is found in libsnd.a

To configure a client application, make sure that the appropriate directories are set at compile time, for example these could be Makefile entries:

	CC -c client.c -I/vss/include/directory
	CC -o client client.o -L/vss/library/directory -lsnd -ll -lm

where /vss/include/directory contains the file vssClient.h
and where /vss/library/directory contains the file libsnd.a
(typically these are the same directory, in fact). Note that the "link command" which builds your client (here, the second line

	CC -o client client.o -L/vss/library/directory -lsnd -ll -lm

The client code must then have

	#include "vssClient.h"

ahead of the function calls. For specific details and a working example see The Trivial Client in the Writing Client Applications section of the reference manual.

The basic client-server connection

	int BeginSoundServer(void);
	void EndSoundServer(void);

These functions return 1 if successful, else they return 0. BeginSoundServer() establishes a UDP connection and handshakes with VSS. EndSoundServer() cleans up memory in VSS used by the client and cleanly breaks the UDP connection.

To establish a connection with VSS running on a specific remote host, use

	int BeginSoundServerAt(char * hostname);

instead of BeginSoundServer(). BeginSoundServer is sufficient for a remote connection to VSS if the server's hostname has already been set as a unix environment variable prior to launching the client application. To do this use the command (from csh)

	setenv SOUNDSERVER hostname

Usage

	if (!BeginSoundServer() )
		exit(2);

Within the exit code of the client (i.e. after normal contact with, and use of, the server), clean up as follows:

	EndSoundServer();

Actor message syntax

Command Arguments;

• The Command calls or refers to the synthesis function
• The Arguments are data passed to the synthesis function; by their nature they depend upon their corresponding Command
• In certain cases of Messages, Arguments may contain other Command and their associated Arguments

The Arguments will typically contain an actor handle (see Actor handles below), followed by one or more parameter-setting values specific to the actor type, so that the message takes the basic form:

Command ActorHandle value [value ...];

	(1) SetPlaybackRate Actorhandle playrate "filename";

	(2) SendBreakpoints Actorhandle envArray;

Usage

	(3) SendBreakpoints my_env [ 0, 0.333, 2, 1, 5, 0 ];

In this example the term "my_env" is a user-specified variable name representing an Actor handle. Brackets are delimiters for array declarations in .aud files.

Actor message arguments may be delimited using any combination of commas and whitespace. The same holds true for delimiting of array elements.

Actor handles

Here is the basic message syntax for messages that return Actor handles:

ActorHandle = Create ActorType;

SoundHandle = BeginSound ActorHandle;

	(4) ThisActor = Create SampleActor
	(5) ThisSound = BeginSound ThisActor, SetAmp amplitude, SetFile "filename";

1. Sound Sources
2. Modifiers
3. Processors
4. Message Groups

Sound Sources are audio signal generators; they produce a signal which is passed to the computer's audio hardware and presented to an external audio signal path, which typically includes an audio amplifier and output transducers such as loudspeakers or headphones. Sound Sources are organized in Sound Groups. A Sound Group is a user-defined collection of sources of a common engine type. A sound group allows a parameter change to be performed synchronously to multiple sound sources of a given type.

Modifiers are functions which modify a control parameter. A Modifier can generate or modify a control signal. An example is an Envelope, a piecewise-linear function applied in timesteps as a scalar to a parameter value, to result in a linear change of the parameter value over a specified duration. Another example is a Mapper which rescales a range of values in a control data stream.

Processors receive an audio signal and perform digital signal processing operations to output a modification of the audio signal. An example is a Directional Mixer which distributes an audio signal to multiple outputs. Another example is a Reverberator which creates time-delayed filtered repetitions of an audio signal.

Message Groups encapsulate data from a client and specify how that data is applied to control sound synthesis. Message Groups provide a structured manner by which events occurring in the client may be tied to Actor Messages in VSS, through the .aud file interface. Thus, Message Groups are used to define and control the nature of the interaction between the running client and the Actors in VSS.

Events occurring in the client are characterized by the computational state of the client and by the interaction between the user and the client. These events may be indexed by time, by value, or both. Through Message Groups, a mapping is established between client-side states (their occurrence in time, and the conditions they represent) and server-side Actors.

Message Groups are the only Actor class directly visible to the client. Message Groups are referred to within the client by their handle name and by the data passed in their argument array. The mapping of client-side events to sound is then constructed external to the client code wherever possible, using Actor Messages within the .aud file.

• Evaluation
  1. evaluate the client application, both the interactivity of the interface and the characteristics of the data.
  2. determine events and information to be represented with sound.
  3. determine the computational states that characterize the events or information. Define these states in terms of numerical representations. These values will be passed from the client to the server via message groups.
  4. call the AUDupdate() function in the client code at each location a specified state can be evaluated. AUDupdate() arguments include the an array of data and the name of a message group to receive and translate the data.
• Interpretation
  1. create an .aud file for the client application. A client can read multiple .aud files, so it is possible to use multiple .aud files to organize the definition of large numbers of message groups.
  2. in the .aud file create a message group for each AUDupdate() in the client program.
  3. consult the menu of sound algorithms provided in the sound server. For each AUDupdate, identify a range of sound which will support an effective representation of a domain of client states.
  4. at the top of the .aud file, add messages to load the required DSOs and initialize instances of the required Actors.
  5. in the .aud file, for each message group define Actor messages to be executed each time the corresponding AUDupdate() is called by the client.
  6. specify the mapping of data from the client array into appropriate message parameters in the Message Group.

Using .aud files a client application does not have to recompile when the mapping of data to sound is altered. This supports rapid prototyping and rapid refinement. "Plug and play" of sound algorithms -- in the form of message groups -- is made possible by borrowing structures from existing .aud files and calling those structures from new client applications. Message groups are an .aud file structure discussed in section 8.

Client-side view of .aud files

	int AUDinit(const char * filename);

	void AUDupdate
	    (int handle, char *MessageGroupName, int Numfloats, float *dataArray);

	void AUDterminate(int handle);

There are three basic steps involved in preparing a client application to communicate with .aud files:

First: Establish client connections and link as described in part II.

	#include "vssClient.h"

	/* Near the top of  C++ client application */
	if (!BeginSoundServer() )
		exit(2);

		/* Here are hypotehtical interaction, simulation and graphic display events */
		while(not_done) {

			interactions = check_control_devices();
			status = update_simulation_states(interactions);
			update_environment_graphics(status);
			not_done = query_status(status);
		}

	/* okay we're getting ready to exit */
	EndSoundServer();

Second: After initializing contact with the server, open one or more .aud files. The .aud file handle returned by AUDinit() allows the client application to access items in more than one .aud file and to clean up VSS memory using AUDterminate() when the client application closes. If AUDinit returns a number less than zero, a syntax error was detected in the .aud file.

	#include "vssClient.h"

	/* create variables for file handles*/
	float handle1, handle2;

	if (!BeginSoundServer() )
		exit(2);

	/* open the audfiles of choice */
	int hContinuous = AUDinit("AUD/continuous_sounds.aud");
	int hConditional = AUDinit("AUD/conditional_sounds.aud");
	if (hContinuous<0 || hConditional<0)
		exit(3);

		while(not_done) {

			interactions = check_control_devices();
			status = update_simulation_states(interactions);
			update_environment_graphics(status);
			not_done = query_status(status);
		}

	/* time to clean up */
	AUDterminate(hContinuous);
	AUDterminate(hConditional);
	EndSoundServer();

Third: use AUDupdate() to make particular sounds when the correct conditions occur. The sounds are created according to a Message Group which is specified in the .aud file. The name of the Message Group is not declared in the client application. The .aud file contains a call which initializes a Message Group of the correct name; this must happen before the client can use that name in an AUDupdate(). When VSS receives the name from the client it looks to see if a Message Group of that name has been initialized.

	#include "vssClient.h"

	float handle1, handle2;

	/* create variables for sending data to message groups */
	float dataArray1[arraySize1], dataArray2[arraySize2];

	if (!BeginSoundServer() )
		exit(2);

	int hContinuous = AUDinit("AUD/continuous_sounds.aud");
	int hConditional = AUDinit("AUD/conditional_sounds.aud");
	if (hContinuous<0 || hConditional<0)
		exit(3);

	while (not_done) {

		interactions = check_control_devices();
		status = update_simulation_states(interactions);
		update_environment_graphics(status);
		not_done = query_status(status);

		/* update and transmit curent state to VSS */

		/* values needed for continuous events*/
		dataArray1 = get_relevant_data(status);

		AUDupdate(hContinuous,
			"OneMessageGroup",
			arraySize1,
			dataArray1);

		/* values needed for special events */
		if (status == alarmcondition)
			dataArray2 = get_special_data(status);

		if (special)
			AUDupdate(hConditional,
				"AnotherMessageGroup",
				arraySize2,
				dataArray2);

		not_done = query_status(status);
	}

	AUDterminate(hContinuous);
	AUDterminate(hConditional);
	EndSoundServer();

The dataArray is an array of floats declared in the client application. The application developer identifies values from the application dynamics or interface state, to be used with specific Message Groups declared in .aud files. The client app places these values in the arrays used in AUDupdate(). Further examples are provided in the Using .aud files section of the VSS reference manual.

Connecting clients to .aud files

In order to create an .aud file you must know what the client application is doing and under what conditions it calls various message groups. You also need to know the order of variables stored in the data arrays passed to message groups. There is no way to automatically tell the .aud file where the variables have come from, just as there is no way to automatically tell the client how the outgoing values are going to be used by the .aud file. When creating a message group you explicitly write the .aud file and the client data arrays to agree with one another. Of course, once the structure and contents of the data arrays is defined in the client, variations of usage can be created in alternative versions of an .aud file, without changing (recompiling) the client.

.aud file command syntax

	handleName = commandName, arg1, arg2,...;

	commandName, arg1, arg2,...;

The simplest .aud files have no message groups

We provide a simple client for playing this sort of .aud file. It is called audTest and it only calls AUDinit(). AUDupdate() is not called; therefore no values are passed interactively to VSS. See the audtest tutorial for introductory examples of .aud file syntax, before studying message groups.

Designating time in .aud files

A Message Group is a datatype declared in an .aud file to manage a set of Actor messages. A Message Group supports the specification of any number of Actor messages listed in a particular order. The list of messages in a Message Group are executed as a group upon passing the Message Group name to an AUDupdate() function in the client application. The syntax is:

	void AUDupdate
	    (int handle, char *MessageGroupName, int Numfloats, float *dataArray);

A valid name corresponds to a Message Group declared in the .aud file.

The array of floats (dataArray) is updated elsewhere by the client application. Then dataArray is passed to AUDupdate(). AUDupdate() passes the updated array to the Actor messages in the message group. This is the most important feature of the message group. The client application does not have to define the usage of the data in dataArray. The assignment of values from dataArray to arguments of the Actor messages is performed in Message Groups in the .aud file .

Because the .aud file defines the usage of the values in dataArray, the client application may send arrays of arbitrary size to AUDupdate(). Values sent to VSS in dataArray that are not needed may simply be ignored by the Message Group in the .aud file.

Structuring client code for AUDupdate()

Some message groups may not require control data from the client application. These messages are called with a null array of size 0:

	AUDupdate(file_handle, "message_with_no_variables", 0, NULL);

Using message groups

	messageHandle = Create MessageGroup;
	AddMessage messageHandle, commandName, args,...;

1. Load DSOs for required Actors
2. Create and name the Actors which will be manipulated through the messages
3. Create and name message groups
4. Send setup messages (like BeginSound or SetFreq) to those Actors
5. Add a list of messages to each message group for the Actor to respond to

The AddMessage command completes the link between the client and the server. The array values passed through the call to AUDupdate(...); are "picked out" here and applied to the Actor referenced by ActorHandle. In this manner, the client can call any Actor message on the list of available commands. "commandName" can be any Actor command.

In the following example .aud file a sample Actor is created and used as an argument to initialize amplitude, to initialize playback sample rate, and to play a soundfile whenever the "Play" message group is called in AUDupdate();

	SampHandle = Create SampleActor;
	Play = Create MessageGroup;
	AddMessage Play SetAmp SampHandle *0;
	AddMessage Play SetPlaybackRate SampHandle *1;
	AddMessage Play PlaySample SampHandle "mysoundfile';

In what order do I put these calls in my CAVE app?

It's best, before firing up the whole CAVE, to verify that the runtime environment is intact (in particular, that VSS is running and your .aud files are okay). Therefore, call BeginSoundServer() and AUDinit() first, before calling any CAVExxx() functions.

Also, note that CAVEExit() actually exits the program, so call AUDterminate() and EndSoundServer() before CAVEExit(), or just call CAVEHalt() instead of CAVEExit()

In other words:

1. BeginSoundServer()
2. AUDinit()
3. CAVEConfigure(), CAVEMalloc(), CAVEInit(), CAVEDisplay()
4. main loop: AUDupdate()'s.
5. AUDterminate()
6. EndSoundServer()
7. CAVEExit()

A few more hints:

Put the libraries -lsnd -lcave -lsphere near the beginning, not near the end, of the link line in your makefile. Certainly put them before low-level things like -lm -lgl. For some reason, you'll get link errors on the SGI if you put libraries "out of order."

Use sginap() and gr_osview to balance cpu load. For a given audio result, you may need to increase the duration of sginap()'s in your client code. Watch gr_osview's CPU usage: as it gets near 100%, you'll hear interruptions in the audio; you may also identify occasions in running your app which demand extra resources. If you have several SGI's, you can tell your app to be a client of several sound servers at once, distribute the sound computation among the machines, and use an analog mixer to combine the machines' audio outputs.

You may prefer to call BeginSoundServer() after CAVEConfigure(), not before, in order to automatically pick up the environment variable $SOUNDSERVER which the cave configuration files can specify for you. Note carefully how this mechanism works, though.

• The cave machine's system.caverc file may specify a default value for $SOUNDSERVER.
• If your app's own .caverc file doesn't specify a different value, system.caverc's value will be passed to your cave app by the call to CAVEConfigure().
• If you need a value other than the default (you're running VSS on a different machine this week, say), then your own .caverc must specify putenv SOUNDSERVER machinename. (The "putenv" is required!)
• We discourage specifying $SOUNDSERVER manually in this context of .caverc files; behavior may be erratic, particularly across different versions of the CAVE libraries.
• If all else fails, add debugging statements to your app like
  "printf("$SOUNDSERVER is currenty <%s>\n\n", getenv("SOUNDSERVER"));.

Caution: most web sounds are very low quality due to reduction of sampling rate and bit rate. Conversion does not upgrade quality once it has been encoded at a lower resolution.

Sound file Library (at NCSA only): /afs/ncsa/projects/blanca/public/sounds
Some of these have not been converted into aiff format.

Soundfile repositories

Finally finally, search the whole web if you can guess the filename someone might have used. Go to Altavista and enter queries like

• +link:.wav +link:beavis
• link:beep*.wav
• link:startrek*.au