TclCommandWriting
Command Writing(TCL) Command Writing(TCL)
NAME
TclCommandWriting - Writing C language extensions to Tcl.
OVERVIEW
This document is intended to help the programmer who wishes to extend
Tcl with C language routines. It should also be useful to someone
wishing to add Tcl to an existing editor, communications program, win-
dow manager, etc. Experienced extension writers may find this manual
helpful in rewriting their applications to use the new Tcl object sys-
tem. We assume you are already fluent in the C programming language
and that you have built and installed Tcl on your machine.
Information on the available C interface routines to Tcl can be found
in the *.3 manual pages in the doc directory of the baseline Tcl dis-
tribution, and in the *.3 manpages in the doc directory of Extended
Tcl.
TCL OBJECT SYSTEM
With the release of Tcl version 8, Tcl has a new system for managing
Tcl values internally. To the Tcl programmer, the new objects look and
act like strings, as before. But at the C level, these objects can now
also hold cached internal representations of the strings in various
native datatypes. For example, an object containing a string consist-
ing of an integer, will now maintain a machine-code integer representa-
tion, if an integer representation has been needed. Using these
objects is much more efficient than using the older-style Tcl strings,
although the older style is still (currently) supported.
Although the object system has almost no effect at all on how the Tcl
programmer uses Tcl, the object system's C interfaces to strings, inte-
gers, lists, etc., have changed considerably. While converting a pack-
age to use the new system can be a lot of work, the combination of the
object system, which saves Tcl from having to constantly convert
strings to integers and back, etc., and the on-the-fly bytecode com-
piler (which keeps Tcl from having to continually reparse code it is to
execute) yield Tcl programs that routinely execute several times more
quickly than with previous versions (Tcl 7 and before), and in some
cases run as much as 2500 (!) times faster than before.
We have chosen, then, to rewrite the Command Writer's manpage, which
has been shipping with Extended Tcl for a number of years, to produce
this new version based on the new object system. The old manpage,
based on the older string-oriented routines, will still be included in
TclX releases for now, as it is still relevant to Tcl releases through
version 7, and may be of use to those modifying/upgrading packages
written for the old model. The old manual will be dropped from the
release once we deem it unneeded; the old interfaces should now be con-
sidered legacy interfaces, and all new development should be done using
the new object interfaces, unless backwards compatibility to pre-Tcl-8
releases is needed.
A SIMPLE C EXTENSION
All C-based Tcl commands are called with four arguments: a client data
pointer, an interpreter pointer, an argument count and a pointer to an
array of Tcl objects containing the arguments to the command.
A simple C extension to Tcl is now presented, and described below:
#include "tcl.h"
int App_DumpArgsObjCmd(clientData, interp, objc, objv)
void *clientData;
Tcl_Interp *interp;
int objc;
Tcl_Obj **objv;
{
int i;
int stringLen;
char *stringPtr;
for (i = 1; i < objc; i++) {
stringPtr = Tcl_GetStringFromObj (objv [i], &stringLen);
printf("%s", stringPtr);
if (i < objc - 1) printf(" ");
}
printf("\n");
return TCL_OK;
}
The client data pointer will be described later.
INTERPRETERS
The interpreter pointer is the ``key'' to an interpreter. It is
returned by Tcl_CreateInterp and is used extensively within Tcl, and
will be used by your C extensions. The data structure pointed to by
the interpreter pointer, and all of the subordinate structures that
branch off of it, make up a Tcl interpreter, which includes all of the
currently defined procedures, commands, variables, arrays and the exe-
cution state of that interpreter. (For more information on creating
and deleting interpreters, please examine the CrtInterp(3) manpage in
the core Tcl distribution. For information on creating interpreters
that include the commands provided by Extended Tcl, check out the
TclX_Init(3) manpage of Extended Tcl. For a manual page describing the
user-visible fields of a Tcl interpreter, please look at Interp(3) in
core Tcl.)
OBJECT COUNT AND ARGUMENTS
The argument count, or object count (objc), and pointer to an array of
pointers to Tcl objects of the command's arguments (objv) is handled by
your C code, in a manner similar to the one you would use in writing a
C main function -- an argument count and array of pointers works the
same as in a C main call; pointers to the arguments to the function are
contained in the objv array. Similar to a C main, the first argument
(objv[0]) is an object containing the name the routine was called as
(in a C main, the name the program was invoked as).
In Tcl, however, the array of pointers are not pointers to character
strings (although they were in all version of Tcl before 8.0).
In the above example, all of the arguments are output with a space
between each one by looping through elements of the objv array from one
to the argument count, objc, and a newline is output to terminate the
line -- a simple ``echo'' command. This example uses printf for sim-
plicity. Of course in production code you would want to use the Tcl
filesystem interfaces. See GetFile(3) and friends for more informa-
tion.
All arguments from a Tcl call to a Tcl C extension are passed as Tcl
Objects. If your C routine wants to look at one of those arguments as
an integer, you need to make a call to a routine to fetch the represen-
tation of the object that you need. In the earlier example, for
instance, Tcl_GetStringFromObj is called to obtain a textual represen-
tation of an object. Additional routines are available to fetch the
representation of a data element as other data types. Tcl_GetBoolean-
FromObj, Tcl_GetDoubleFromObj, Tcl_GetIntFromObj, Tcl_GetLongFromObj,
and Tcl_GetIndexFromObj, fetch object representations of Tcl strings as
booleans, double-precision floating point, integer, long integer, and
lists, among others.
These routines automatically leave an appropriate error message in the
Tcl interpreter's result object and return TCL_ERROR if a conversion
error occurs. (For more information on these routines, please look at
the Object(3) manpage in the core Tcl distribution.)
RETURNING RESULTS
As you might expect, the API for setting results from C extensions has
changed significantly under the object system. The old technique of
writing small results directory into the interpreter's result buffer is
no longer used, for example. The notion of having to tell Tcl whether
a result is static or dynamic is also a thing of the past. Under the
object system, results are objects that are set up by your code, and
objects are freed when their reference counts say they should be. More
on this later.
If you program produces a numeric result, it should set the result
object to contain that numeric value. A common way of doing this is
something like...
Tcl_Obj *obj;
obj = Tcl_GetObjResult (interp);
Tcl_SetIntObj (obj, value);
The above code obtains a pointer to the result object (an object made
available to your routine that you're supposed to store your results
into) and sets the integer value value into it.
Another way to do it would be to set up a new object and tell Tcl that
this object contains the result...
Tcl_Obj *resultObj;
/* create a new object for use as a result */
resultObj = Tcl_NewObj ();
Tcl_SetIntObj (obj, value);
Tcl_SetObjResult (interp, resultObj);
Understanding how results are passed back to Tcl is essential to the C
extension writer. Please study the SetObjResult(3) manual page in the
Tcl distribution for more information.
VALIDATING ARGUMENTS
It is a design goal of Tcl that no Tcl program be able to cause Tcl to
dump core. It is important that the extension writers, likewise, use
the avaiable methods and tools to make sure that their extensions do
not allow unchecked input, for example, to cause the code to get some
kind of runtime exception.
The object system has simplified, to some degree, the task of validat-
ing arguments, in that the object system automatically attempts type
conversions as needed, and will return an error when a type conversion
fails.
A simple, but important, check that every C extension should do is ver-
ify that it has the right number of arguments.
The act of trying to use, say, a string as an integer, implicitly per-
forms the type conversion of the string and, if it doesn't work as an
integer, returns TCL_ERROR. The developer should check for the
TCL_ERROR return from all of the GetXxxFromObj commands, and handle
them as appropriate. Usually this will mean propagating the error on
back to the user, or to an intevening catch, as the case may be.
You should also check that values are in range (when their ranges are
known), and so forth. When C data structures need to be handled in Tcl
in some form or another, yet the contents of the data must remain
opaque to Tcl, as is usually the case with binary data (although
futures releases of Tcl are expected to have native abilities to read,
write and manipulate binary data instrinsically), handles need to be
used. Handles will be described and examples presented, later in this
doc.
ANOTHER C EXTENSION - THE MAX COMMAND
In the command below, two or more arguments are compared, and the one
with the maximum value is returned, if all goes well. It is an error
if there are fewer than two arguments (the pointer to the ``max'' com-
mand text itself, objv[0], and a pointer to at least one object to com-
pare the values of).
int
Tcl_MaxCmd (clientData, interp, objc, objv)
char *clientData;
Tcl_Interp *interp;
int objc;
Tcl_Obj **objv;
{
int maxVal = MININT;
int value, idx;
if (objc < 3)
return TclX_WrongArgs (interp, objv[0],
" num1 num2 [..numN]");
for (idx = 1; idx < objc; idx++) {
if (Tcl_GetIntFromObj (interp, objv[idx], &value) != TCL_OK)
return TCL_ERROR;
if (value > maxVal) {
maxVal = value;
}
}
Tcl_SetIntObj (Tcl_GetObjResult (interp), value);
return TCL_OK;
}
Here we introduce the Extended Tcl helper function TclX_WrongArgs.
This routine makes it easy to create an error message and error return
in response to the common mistake of being called with a wrong number.
Tcl_GetIntFromObj is used to fetch the integer values of the remaining
arguments. If any fail to be converted, we return a Tcl error. If an
interpreter is specified in the call to Tcl_GetIntFromObj, an appropri-
ate error message about the conversion failure will be left in the
result, so we do that here.
After examining all of the arguments to find the largest value, we set
the result object to contain that value, and return TCL_OK.
RETURNING RESULTS
When Tcl-callable functions complete, they should normally return
TCL_OK or TCL_ERROR. TCL_OK is returned when the command succeeded,
and TCL_ERROR is returned when the command has failed in some abnormal
way. TCL_ERROR should be returned for all syntax errors, non-numeric
values when numeric ones were expected, and so forth. Less clear in
some cases is whether Tcl errors should be returned or whether a func-
tion should just return a status value. For example, end-of-file dur-
ing a gets returns a status, but open returns an error if it fails.
Errors can be caught from Tcl programs using the catch command. (See
Tcl's catch(n) and error(n) manual pages.)
Less common return values are TCL_RETURN, TCL_BREAK and TCL_CONTINUE.
These are used if you are adding new control and/or looping structures
to Tcl. To see these values in action, examine the source code to
Extended Tcl's loop commands. Tcl's while, for and if commands used to
work in the just same manner, but are now compiled into bytecode by the
bytecode for performance.
ANOTHER C EXTENSION - THE LREVERSE COMMAND
In the command below, a list is passed as an argument, and a list con-
taining all of the elements of the list in reverse order is returned.
It is an error if anything other than two arguments are passed (the
pointer to the ``lreverse'' command text itself, objv[0], and a pointer
to the list to reverse.
Once lreverse has determined that it has received the correct number of
arguments, Tcl_ListObjGetElements is called to split the list into its
own objc count of elements and objv array of pointers to the list's
elements.
lreverse then operates on the array of pointers, swapping them from
lowest to highest, second-lowest to second-highest, and so forth.
Tcl_ListObjAppendElement is called on successive list elements to build
up the new list, which is finally returned as result of the command.
int
Tcl_LreverseObjCmd(notUsed, interp, objc, objv)
ClientData notUsed; /* Not used. */
Tcl_Interp *interp; /* Current interpreter. */
int objc; /* Number of arguments. */
Tcl_Obj **obj; /* Argument strings. */
{
int listObjc, lowListIndex, hiListIndex;
Tcl_Obj **listObjv;
char *temp, *resultList;
Tcl_Obj **newListObjv;
/* Verify argument count. Since we take only one argument, argument
* count must be 2 (command plus one argument).
*/
if (objc != 2)
return TclX_WrongArgs (interp, objv [0], "list");
/* Create an object to handle the new list we're creating */
newListObjv = Tcl_NewObj();
/* Crack the list at objv[1] into its own count and array of object
* pointers.
*/
if (Tcl_ListObjGetElements (interp, objv[1], &listObjc, &listObjv) != TCL_OK) {
return TCL_ERROR;
}
/* For each element in the source list from last to first, append an
* element to the new list.
*/
for (listIndex = listObjc - 1; listIndex >= 0; listIndex--) {
Tcl_ListObjAppendElement (interp, newListObjv, listObjv[listIndex]);
}
FIX: NEED TO RETURN THE LIST.
return TCL_OK;
}
INSTALLING YOUR COMMAND
To install your command into Tcl you must call Tcl_CreateObjCommand,
passing it the pointer to the interpreter you want to install the com-
mand into, the name of the command, a pointer to the C function that
implements the command, a client data pointer, and a pointer to an
optional callback routine.
The client data pointer and the callback routine will be described
later.
For example, for the max function above (which, incidentally, comes
from TclX's tclXmath.c in the TclX7.4/src directory):
Tcl_CreateCommand (interp, "max", Tcl_MaxCmd, (ClientData)NULL,
(void (*)())NULL);
In the above example, the max function is added to the specified inter-
preter. The client data pointer and callback function pointer are
NULL. (For complete information on Tcl_CreateCommand and its companion
routine, Tcl_CommandInfo, please examine the CrtCommand(3) command page
in the core Tcl distribution.)
DYNAMIC STRINGS
Dynamic strings are an important abstraction that first became avail-
able with Tcl 7.0. Dynamic strings, or DStrings, provide a way to
build up arbitrarily long strings through a repeated process of append-
ing information to them. DStrings reduce the amount of allocating and
copying required to add information to a string. Further, they sim-
plify the process of doing so.
At first glance, it may seem that the object system supersedes
DStrings. It does not, in that the performance improvements made pos-
sible by the lazy conversion of an object's representation from one
datatype to another does not come into play much while constructing
strings as the string representation is always available either without
any type conversion or where type conversion would be necessary in any
case as a string representation of the object is required when strings
are being constructed by concatenation, etc.
It should be noted, however, that the C level string manipulation capa-
bilites of objects, such as Tcl_AppendToObj and Tcl_AppendStringsToObj,
are often plenty enough for what you need to do. For complete informa-
tion on dynamic strings, please examine the DString(3) manual page in
the core Tcl distribution. For more on Tcl object's string-oriented
calls, seek Tcl_StringObj(3) in the same location.
CLIENT DATA
The client data pointer provides a means for Tcl commands to have data
associated with them that is not global to the C program nor included
in the Tcl core. Client data is essential in a multi-interpreter envi-
ronment (where a single program has created and is making use of multi-
ple Tcl interpreters) for the C routines to maintain any permanent data
they need on a per-interpreter basis. If needed static data was simply
declared static in C, you will probably have reentrancy problems when
you work with multiple interpreters.
Tcl solves this through the client data mechanism. When you are about
to call Tcl_CreateObjCommand to add a new command to an interpreter, if
your command needs to keep some read/write data across invocations, you
should allocate the space, preferably using Tcl_Alloc instead of
malloc, then pass the address of that space as the ClientData pointer
to Tcl_CreateObjCommand.
When your command is called from Tcl, the ClientData pointer you passed
to Tcl_CreateObjCommand will be passed to your C routine through the
ClientData pointer calling argument.
Commands that need to share this data with one another can do so by
using the same ClientData pointer when the commands are added.
It is important to note that the Tcl extensions in the tclX8.0.0 direc-
tory have had all of their data set up in this way. Since release 6.2,
Extended Tcl has supported multiple interpreters within one invocation
of Tcl.
THEORY OF HANDLES
Sometimes you need to have a data element that isn't readily repre-
sentable as a string within Tcl, for example a pointer to a complex C
data structure. It is not a good idea to try to pass pointers around
within Tcl as strings by converting them to and from hex or integer
representations, for example. It is too easy to mess one up, and the
likely outcome of doing that is a core dump.
Instead we have developed and made use of the concept of handles. Han-
dles are identifiers a C extension can pass to, and accept from, Tcl to
make the transition between what your C code knows something as and
what name Tcl knows it by to be as safe and painless as possible. For
example, the I/O system included in Tcl uses file handles. When you
open a file from Tcl, a handle is returned of the form filen where n is
a file number. When you pass the file handle back to puts, gets, seek,
flush and so forth, they validate the file handle by checking the the
file text is present, then converting the file number to an integer
that they use to look into a data structure of pointers to Tcl open
file structures, which contain a Unix file descriptor, flags indicating
whether or not the file is currently open, whether the file is a file
or a pipe and so forth.
Handles have proven so useful that, since TclX release 6.1a, general
support has been available to help create and manipulate them. Many of
these capabilities have migrated into baseline Tcl. If you have a sim-
ilar need, you might like to use the handle routines documented in Han-
dles(3) in Extended Tcl. We recommend that you use a unique-to-your-
package textual handle coupled with a specific identifier and let the
handle management routines validate it when it's passed back. It is
much easier to track down a bug with an implicated handle named some-
thing like file4 or bitmap6 than just 6.
Note that Tcl's object offers another way for complex data structures
to exist in parallel with and underneath Tcl strings. As of this writ-
ing (May 30, 1997) this is fairly new territory, but things are looking
good for the prospects of using the Tcl object system in this manner,
and for enhancements to the object system that allow even Tcl objects
to have methods in a very straightforward and simple way.
USING COMMANDS TO DO THE SAME THING, AND MORE
Another handle-like technique, first popularized in the Tk toolkit,
offers handle-like capabilities as well as some neat additional capa-
bilities. That is to create a new Tcl command, from C, that uses
ClientData to keep a "handle" on its complex underlying data structure.
Then by having that command look at its second argument for what it is
to do (its sub-functions), you get these nice methods, where you have
several additional sub-commands that don't pollute the global namespace
and only work on (and are available with) the objects (new commands)
they are relevant to. For example, in Tk, creating a checkbutton
(checkbutton .b) creates a new Tcl command (.b), that has methods to
configure the button, select, deselect, toggle and flash it.
A lot of people think this is really the way to go, and I am pretty
much leaning that way myself. If you use the incr tcl script-level
object system for Tcl, objects that you define in Tcl will be highly
compatible in terms of their command interfaces and configuration man-
agement with objects you create in C using the the command-and-Client-
Data technique described here. I believe Tk has some nice facilities
for making this easy for the Tcl programmer. Itcl certainly does.
TRACKING MEMORY CORRUPTION PROBLEMS
Occasionally you may write code that scribbles past the end of an allo-
cated piece of memory. This will usually result in a core dump or mem-
ory allocation failure sometime later in the program, often implicating
code that is not actually responsible for the problem (as you start
looking from the point where the error is detected, which is usually
where the later routine has failed).
The memory debugging routines included in Tcl can help find these prob-
lems. Developed by Mark and Karl, the memory debugging routines are
now part of baseline Tcl, and is to our knowledge the largest piece of
TclX to drop into the core without being reengineered first. (You see,
summer back in '91, John was sitting in his office in the CS building
at UC Berkeley trying to find a memory leak somewhere in Tcl, when he
was paid a visit by two long-haired-yet-polite programmers bearing
gifts in the form of the technology grab-bag known as Extended Tcl. He
saw that, using TclX's malloc routines, Tcl could be prompted to print
the filename and line number of every single memory allocation that did
not have a corresponding free. It was just what the doctor ordered ;-)
See Memory(TCL) for details.
INSTALLING YOUR EXTENSIONS INTO TCL
To add your extensions to Tcl, you used to have to statically link
them, together with any other extensions, into a single binary exe-
cutable image. Today, although the statically linked executable is
still an option, most operating systems, even Microsoft Windows, sup-
port shared libraries, and in most cases, Tcl can now make use of those
shared libraries such that you extensions, and most others, can now be
built a shared libraries that can be loaded in (using package require)
by scripts that need them. Shared libraries can simplify a Tcl instal-
lation, because only one copy of Tcl is required, rather than a hode-
podge of combinations of applications that you might have found at a
big Tcl site in the previous era.
GNU AUTOCONF
While the build procedure for shared libraries varies from system to
system, most Unix and Unix workalike systems will figure out the
nuances of the compiler and linker arguments automatically when the
configure script is run. If you are building a package that you plan
to make generally available, we strongly recommend that you use GNU
autoconf (ftp://prep.ai.mit.edu/pub/gnu) to set up an automatic config-
ure script for it. Be forewarned that autoconf uses some pretty heavy
duty shell and sed script magic to get the job done, and the learning
curve can be pretty steep. Once done and shaken out, though, it's
rewarding to know that your package can build and run on everything
from a notebook to a Cray to a RISC SMP server.
Application-specific startup is accomplished by creating or editing the
Tcl_AppInit function. In Tcl_AppInit you should add a call to an
application-specific init function which you create. This function
should take the address of the interpreter it should install its com-
mands into, and it should install those commands with Tcl_CreateCommand
and do any other application-specific startup that is necessary.
The naming convention for application startup routines is App_Init,
where App is the name of your application. For example, to add an
application named cute one would create a Cute_Init routine that
expected a Tcl_Interp pointer as an argument, and add the following
code to Tcl_AppInit:
if (Cute_Init (interp) == TCL_ERROR) {
return TCL_ERROR;
}
As you can guess from the above example, if your init routine is unable
to initialize, it should use Tcl_AppendResult to provide some kind of
useful error message back to TclX, then return TCL_ERROR to indicate
that an error occurred. If the routine executed successfully, it
should return TCL_OK.
When you examine Tcl_AppInit, note that there is one call already there
to install an application -- the call to TclX_Init installs Extended
Tcl into the Tcl core.
MAKING APPLICATION INFORMATION VISIBLE FROM EXTENDED TCL
TclX's infox command can return several pieces of information relevant
to Extended Tcl, including the application's name, descriptive name,
patch level and version. Your application's startup can set these
variables to application-specific values. If it doesn't, they are
given default values for Extended Tcl.
To set these values, first be sure that you include either tclExtend.h
or tclExtdInt.h from the source file that defines your init routine.
This will create external declarations for the variables. Then, set
the variables in your init route, for example:
tclAppName = "cute";
tclAppLongName = "Call Unix/Tcl Environment";
tclAppVersion = "2.1";
Note that the default values are set by TclX_Init, so if you wish to
override them, you must call your init routine in Tcl_AppInit after its
call to TclX_Init.
EXTENDED TCL EXIT
When Extended Tcl exits, Tcl_DeleteInterp may be called to free memory
used by Tcl -- normally, this is only called if TCL_MEM_DEBUG was
defined, since Unix will return all of the allocated memory back to the
system, anyway. If TCL_MEM_DEBUG was defined, it is called so that any
memory that was allocated without ever being freed can be detected.
This greatly reduces the amount of work to detect and track down memory
leaks, a situation where some piece of your code allocates memory
repeatedly without ever freeing it, or at least without always freeing
it.
It is often necessary for an application to perform special cleanup
functions upon the deletion of an interpreter as well. To facilitate
this activity, Tcl provides the ability to perform a function callback
when an interpreter is deleted. To arrange for a C function to be
called when the interpreter is deleted, call Tcl_CallWhenDeleted from
your application initialization routine. For details on how to use
this function, read the CallDel(3) manual page that ships with core
Tcl.
EXECUTING TCL CODE FROM YOUR C EXTENSION
Suppose you are in the middle of coding a C extension and you realize
that you need some operation performed, one that would be simple from
Tcl, but possibly excruciating to do directly in C. Tcl provides a
number of C-level interfaces whereby you can cause Tcl code to be exe-
cuteed. The old-style calls are Tcl_Eval, Tcl_VarEval, Tcl_EvalFile
and Tcl_GlobalEval. The results of these calls can be dug out of the
interpreter using Tcl_GetStringResult, if you want a string representa-
tion of the result, or Tcl_GetObjResult if you want the object. (The
use of interp->result to access the result string has been deprecated.)
The Tcl object system adds Tcl_EvalObj and Tcl_GlobalEvalObj. The dif-
ference here is that we are evaluating an object, not just a string,
and using these routines in preference to the aforementioned ones can
result in a major performance improvement in your code, when the code
is executed repeatedly (even if it only executes once but loops several
times within itself), as these routines make it possible for the byte-
code compiler to compile the code being evaluated and save the compiled
code with the data structure, in an implementation-dependent manner.
For more information please consult the EvalObj(3) and Eval(3) manual
pages within the Tcl distribution.
ACCESSING TCL VARIABLES AND ARRAYS FROM YOUR C EXTENSIONS
In addition to the non-object-system ways of reading from and storing
to Tcl variables, using routines such as Tcl_SetVar2 and Tcl_GetVar2,
Tcl variables and arrays can be read from a C extension as Tcl objects
by using the Tcl_ObjGetVar2 function, and set from C extensions through
the Tcl_ObjSetVar2 function.
Please note that the object versions do not carry forward analogues to
the one-variable-name-argument Tcl_GetVar, Tcl_SetVar, and Tcl_Unset-
Var. If you know you have a scalar, call the object variable get and
set functions with a NULL second argument. If your variable name might
contain an array reference via a self-contained embedded array index
(i.e., I'm asking Tcl_ObjGetVar2 for "foo(5)" instead of "foo" "5"),
add the TCL_PARSE_PART1 to the flags in your call.
While the fact that Tcl_ObjGetVar2 retrieves Tcl objects, rather than
strings, is critical for the object system to be able to provide the
performance boosts from "lazy" type conversion and the binary data
capabilities, the arguments containing the variable name, or the array
name and element name if they've been split out, also must be specified
as Tcl objects rather than strings. While this is useful on occasion,
those writing C extensions for Tcl in the post-object-system era usu-
ally have the names available as plain old char * variables, requiring
conversion of the strings to objects before use and account for their
possible destruction afterwards.
To simplify the task in those cases, TclX adds the TclX_ObjGetVar2S
subroutine. It works just like Tcl_ObjGetVar2, except the one or two
variable name arguments are specified as strings, and the routine takes
care of making and disposing of object equivalents.
Tcl variables can be unset from C via the Tcl_UnsetVar and Tcl_Unset-
Var2 functions. There are currently (as of 8.0) no object-system
equivalents, so in the rare case where you have the name of the vari-
able you want unset as an object instead of a string, you can call
Tcl_GetStringFromObj to obtain the string representation first.
For complete information on these functions, please refer to the
ObjSetVar(3) and SetVar(3) manual pages in the doc directory of the
core Tcl distribution.
LINKING TCL VARIABLES TO C VARIABLES
Tcl_LinkVar and Tcl_UnlinkVar can be used to automatically keep Tcl
variables synchronized with corresponding C variables. Once a Tcl
variable has been linked to a C variable with Tcl_LinkVar, anytime the
Tcl variable is read, the value of the C variable is converted (if nec-
essary) and returned, and when the Tcl variable is written, the C vari-
able will be updated with the new value.
Tcl_LinkVar uses variable traces to keep the Tcl variable named by var-
Name in sync with the C variable at the address given by addr.
Int, double, boolean and char * variables are supported. You can make
your linked variables read only from the Tcl side, as well. Please
note that the C variables must continually exist while they are linked,
in other words, linking "automatic" C variables, those created on the
stack while a routine is being executed and destroyed afterwards, will
result in a malfunctioning program at best and a coredump or more at
worst.
For more information, please examine the LinkVar(3) manual page in the
core Tcl distribution.
ADDING NEW MATH FUNCTIONS TO TCL
As of Tcl version 7.0, math functions such as sin, cos, etc, are
directly supported within Tcl expressions. These obsolete the Extended
Tcl commands that provided explicit commands for these functions for
many, many releases, although procs equivalencing the old TclX commands
to the new math functions are still provided for backwards compatibil-
ity.
New math functions can be added to Tcl, or existing math functions can
be replaced, by calling Tcl_CreateMathFunc.
ACCESSING AND MANIPULATING THE RANDOM NUMBER GENERATOR
Prior to Tcl version 8.0, the Tcl core did not provide access to a ran-
dom number generator, but TclX did, through its random command. As of
Tcl version 8.0, access to a random number generator is provided by
baseline Tcl through the new math functions, rand and srand.
The TclX random command is still available -- it has some useful capa-
bilities not directly provided by the new baseline functions.
For more information on adding your own math functions to Tcl, please
study the CrtMathFnc(3) manual page in the core Tcl distribution.
CONVERTING FILENAMES TO NATIVE FORM AND PERFORMING TILDE SUBSTITUTIONS
The Tcl_TranslateFileName function is available to C extension writers
to translate filenames to a form suitable for use by the local operat-
ing system. It converts network names to their native form, and if the
name starts with a ``~'' character, the function returns a new string
where the name is replaced with the home directory of the given user.
For more information please consult the Translate(3) manual page in the
core Tcl distribution.
SETTING THE RECURSION LIMIT
Tcl has a preset recursion limit that limits the maximum allowable
nesting depth of calls within an interpreter. This is useful for
detecting infinite recursions before other limits such as the process
memory limit or, worse, available swap space on the system, run out.
The default limit is just a guess, however, and applications that make
heavy use of recursion may need to call Tcl_SetRecursionLimit to raise
this limit. For more information, please consult the SetRecLmt(3) man-
ual page in the core Tcl distribution.
HANDLING SIGNALS FROM TCL EXTENSIONS
If an event such as a signal occurs while a Tcl script is being exe-
cuted, it isn't safe to do much in the signal handling routine -- the
Tcl environment cannot be safely manipulated at this point because it
could be in the middle of some operation, such as updating pointers,
leaving the interpreter in an unreliable state.
The only safe approach is to set a flag indicating that the event
occurred, then handle the event later when the interpreter has returned
to a safe state, such as after the current Tcl command completes.
The Tcl_AsyncCreate, Tcl_AsyncMark, Tcl_AsyncInvoke, and
Tcl_AsyncDelete functions provide a safe mechanism for dealing with
signals and other asynchronous events. For more information on how to
use this capability, please refer to the Async(3) manual page in the
core Tcl distribution.
Note that Extended Tcl provides built-in support for managing signals
in numerous ways, including generating them with alarm(2) and kill(2),
ignoring them, trapping them, getting, setting, blocking and unblocking
them. You can cause specific code to execute at a safe point after a
signal occurs, or cause a Tcl error backtrace on one's occurrence. For
more information, please examine the TclX documentation.
PARSING BACKSLASH SEQUENCES
The Tcl_Backslash function is called to parse Tcl backslash sequences.
These backslash sequences are the usual sort that you see in the C pro-
gramming language, such as \n for newline, \r for return, and so forth.
Tcl_Backslash parses a single backslash sequence and returns a single
character corresponding to the backslash sequence.
For more info on this call, look at the Backslash(3) manual page in the
core Tcl distribution. For information on the valid backslash
sequences, consult the summary of Tcl language syntax, Tcl(n) in the
same distribution.
HASH TABLES
Hash tables provide Tcl with a high-performance facility for looking up
and managing key-value pairs located and maintained in memory. Tcl
uses hash tables internally to locate procedure definitions, Tcl vari-
ables, array elements, file handles and so forth. Tcl makes the hash
table functions accessible to C extension writers as well.
Hash tables grow automatically to maintain efficiency, rather than
exposing the table size to the programmer at allocation time, which
would needlessly add complexity to Tcl and would be prone to ineffi-
ciency due to the need to guess the number of items that will go into
the table, and the seemingly inevitable growth in amount of data pro-
cessed per run over the useful life of the program.
For more information on hash tables, please consult the Hash(3) manual
page in the core Tcl distribution.
TRACING VARIABLE ACCESSES
The C extension writer can arrange to have a C routine called whenever
a Tcl variable is read, written, or unset. Variable traces are the
mechanism by which Tk toolkit widgets such as radio and checkbuttons,
messages and so forth update without Tcl programmer intervention when
their data variables are changed. They are also used by the routine
that links Tcl and C variables, Tcl_LinkVar, described above.
Tcl_TraceVar is called to establish a variable trace. Entire arrays
and individual array elements can be traced as well. If the programmer
already has an array name in one string and a variable name in another,
Tcl_TraceVar2 can be called. Calls are also available to request
information about traces and to delete them.
For more information on variable traces, consult the TraceVar(3) manual
page in the core Tcl distribution.
TRACING TCL EXECUTION
Tcl has the ability to call C routines each time it executes a Tcl com-
mand, up to a specified depth of nesting levels. The command Tcl_Cre-
ateTrace creates an execution trace; Tcl_DeleteTrace deletes it.
Command tracing is used in Extended Tcl to implement the cmdtrace Tcl
command, a useful command for debugging Tcl applications.
For complete information on execution tracing, please look at the Crt-
Trace(3) manual pages in the core Tcl distribution.
EVALUATING TCL EXPRESSIONS FROM C
Tcl_ExprLong, Tcl_ExprDouble, Tcl_ExprBool, and Tcl_ExprString all take
string arguments and, when called, evaluate those strings as Tcl
expressions. Depending on the routine called, the result is either a C
long, a double, a boolean (int with a value of 0 or 1), or a char *
(obtainable through Tcl_GetResult).
To take advantage of the performance gains available through the byte-
code compiler, Tcl_ExprLongObj, Tcl_ExprDoubleObj, Tcl_ExprBoolObj, and
Tcl_ExprObj all take an object containing an expression to be evaluated
(rather than a string.) The result is that bytecode-compiled version
of the expression will be kept in the object, alongside the string rep-
resentation. If the expression is evaluated again, without being
changed, it does not have to be recompiled... a major performance win.
For complete information on evaluating Tcl expressions from C, you are
invited to examine the ExprLong(3) and ExprLongObj(3) manpages in the
core Tcl distribution.
PATTERN MATCHING
The Tcl_StringMatch function can be called to see if a string matches a
specified pattern. Tcl_StringMatch is called by the Tcl string match
command, so the format for patterns is identical. The pattern format
is similar to the one used by the C-shell; string(n) describes this
format.
More information about Tcl_StringMatch is available in the StrMatch(3)
manpage in the core Tcl distribution.
REGULAR EXPRESSION PATTERN MATCHING
Tcl_RegExpMatch can be called to determine whether a string matches a
regular expression. Tcl_RegExpMatch is used internally by the regexp
Tcl command.
As regular expressions are typically "compiled" before use, a fairly
involved process, Tcl also supports routines that separate the compila-
tion of an expression from its use: Tcl_RegExpCompile, Tcl_RegExpExec,
and Tcl_RegExpRange. If an expression is going to be matched many
times, doing the compile once and caching the compiled regular expres-
sion result, then reusing the cached version by using Tcl_RegExpExec,
can be a significant performance win.
For more information on this function, please consult the RegExp(3)
manpage in the core Tcl distribution.
MANIPULATING TCL LISTS FROM C EXTENSIONS
The C extension writer often needs to create, manipulate and decompose
Tcl lists. Tcl_SplitList and Tcl_Merge used to be the only way to
parse strings into lists and vice versa. As of Tcl 8, lists can be
parsed and assembled, object-style, using Tcl_ListObjGetElements and
Tcl_SetListObj, and friends. Once again the "win" of using object-sys-
tem-based list manipulation, instead of the previous string based rou-
tines, is that the parsing of a string in an object to a list is cached
in the object structure, the same as with integers and floating point
numbers, compiled procedures, etc. The next time this string needs to
be looked at as a list, if the contents of the string have not changed,
the string does not have to be parsed.
In the author's experience, working with an admittedly degenerate test
whereby we iterated rather inefficiently across a 6,000-element list, a
speedup factor of more than 2500 was obtained over the previous non-
object-based version of Tcl.
For more information on these commands, please consult the ListObj(3)
manual page in the core Tcl distribution.
CONCATENATING STRINGS
Tcl_ConcatObj concatenates the string representation of zero or more
objects into a single new object. The elements of the new string are
space-separated. Tcl_Concat does the same thing for strings, as
Tcl_ConcatObj does for objects.
Concatenating strings is similar to constructing lists from them,
except that Tcl_ConcatObj and Tcl_Concat do not attempt to make the
resulting string into a valid Tcl list.
Tcl_Concat is documented in the Concat(3) manpage, and Tcl_ConcatObj in
the tringObj manpage, both in the core Tcl distribution.
DETECTING WHETHER OR NOT YOU HAVE A COMPLETE COMMAND
C routines that collect data to form a command to be passed to Tcl_Eval
often need a way to tell whether they have a complete command already
or whether they need more data. (Programs that read typed-in Tcl input
such as Tcl shells need this capability, for instance.) Tcl_Command-
Complete can be used to tell whether or not you have a complete com-
mand.
For more information examine CmdCmplt(3) in the core Tcl distribution.
RECORDING COMMANDS FOR COMMAND HISTORY
Tcl has a history mechanism that is accessed from Tcl through the his-
tory command. If you want your extension to propagate commands into
the command history, you should call Tcl_RecordAndEvalObj (object sys-
tem) or Tcl_RecordAndEval (old system),
These commands work like Tcl_EvalObj and Tcl_Eval, respectively, except
that these versions record the command as well as executing it.
Tcl_RecordAndEval and Tcl_RecordAndEvlObj should only be called with
user-entered top-level commands, since the history mechanism exists to
allow the user to easily access, edit and reissue previously issued
commands.
For complete information on these functions, please examine the
RecordEval.3 and RecEvalObj.3 manual pages in the core Tcl distribu-
tion.
CONVERTING FLOATING POINT VALUES TO STRINGS
The Tcl object system's Tcl_GetDoubleFromObj and Tcl_SetDoubleObj use
Tcl objects, rather than the strings used by Tcl_PrintDouble, and con-
vert, when necessary, an ASCII string to a double and back again.
These routines ensure that the string output will continue to be inter-
pretable as a floating point number, rather than an integer, by always
putting a ``.'' or ``e'' into the string representing the number.
The precision of the output string is controlled by the Tcl tcl_preci-
sion variable.
For complete information on these routines, please examine DoubleObj(3)
and PrintDbl(3) in the core Tcl distribution.
CREATING CHILD PROCESSES AND PIPELINES FROM C
Tcl_OpenCommandChannel provides a C-level interface to the exec and
open commands. The child (or pipeline of children) can have its stan-
dard input, output and error redirected from files, variables or pipes.
To understand the meaning of the redirection symbols understood by this
function, look at the exec(n) Tcl command. For complete information on
Tcl_OpenCommandChannel, please examine OpenFileChnl(3).
ACCESSING TCL FILEHANDLES FROM C
On Posix/Unix systems, Tcl filehandles passed to your C extension can
be translated to a Posix FILE * structure using the Tcl_GetOpenFile
function, documented in GetOpnFl.3.
MANAGING BACKGROUND PROCESS TERMINATION AND CLEANUP
When a Posix system does a fork to create a new process, the process ID
of the child is returned to the caller. After the child process exits,
its process table entry (and some other data associated with the pro-
cess) cannot be reclaimed by the operating system until a call to wait-
pid, or one of a couple of other, similar system calls, has been made
by the parent process.
The C extension writer who has created a subprocess, by whatever mecha-
nism, can turn over responsibility for detecting the processes' termi-
nation and calling waitpid to obtain its exit status, by calling
Tcl_DetachPids on it.
Tcl_ReapDetachedProcs is the C routine that will detect the termination
of any processes turned over to Tcl, permitting the processes to be
fully reclaimed by the operating system. It is usually not necessary
to call Tcl_ReapDetachedProcs, as it is called automatically every time
exec is performed.
For complete information on these routines, please look at Detach-
Pids(3) in the core Tcl distribution.
FOR MORE INFORMATION
In addition to the documentation referenced above, you can learn a lot
by studying the source code of the commands added by Tcl, Tk and
Extended Tcl, etc. The comp.lang.tcl Usenet newsgroup is read by hun-
dreds of thousands of Tcl people. A number of Frequently Asked Ques-
tions (FAQs) about Tcl are posted there periodically. The newsgroup is
a good place to ask questions (after you've made sure they're not
already answered in the FAQ ;-)
Finally, if you have interactive Internet access, you can ftp to
ftp://ftp.neosoft.com/pub/tcl, the site for contributed Tcl sources.
This site contains quite a few extensions, applications, and so forth,
including several object-oriented extension packages.
If you have access via the world-wide web, check out the Sun Microsys-
tems site (http://sunscript.sun.com), the contributed sources archive
website (http://www.neosoft.com/tcl), and the homepage for Extended Tcl
(http://www.neosoft.com/tclx).
AUTHORS
Extended Tcl was created by Karl Lehenbauer (karl@neosoft.com) and Mark
Diekhans (markd@grizzly.com).
Tcl Command Writing(TCL)
