TIP: | 453 |
Title: | Tcl Based Automation for tcl/pkgs |
Version: | $Revision: 1.2 $ |
Author: | Sean Woods <yoda at etoyoc dot com> |
State: | Draft |
Type: | Project |
Tcl-Version: | 8.7 |
Vote: | Pending |
Created: | Tuesday, 13 September 2016 |
Keywords: | Build tooling |
This TIP proposes replacing the make package process currently employed by the core with a Tcl-based build automation tool.
TIP #376 provides for the distribution of third party packages in the Tcl/Tk core distributions. To support that TIP currently requires three separate build automations: a Makefile based automation in Unix, and both an nmake and Makefile based automation for Windows. These automation systems can get out of sync and they assume that their job is to build dynamic libraries for local installation.
By the time make packages has run, the local Tcl interpreter has been built already. Rather than rely on delicate hacks and makefile tricks, core distributed packages could be built and installed via exec commands inside a Tcl script. In addition, this same automation could handle functions like injecting a core distributed package into a virtual file system, as well as bundling the Tcl/Tk library file system for TIP #430.
For the practcl branch of tclconfig, I put together a 4000 line self-contained package and kit building library of tools. This library is TclOO based, and provides a rudimentary (but functional) system for templating C code in Tcl, as well as a build system that is capable of nesting sub-projects. It also steals the useful bits from the fileutil module of tcllib, providing implementations for concatenating files, performing file searches, and building a global package index from a soup of modules. The library also has a wrapper to download external sources from fossil. It also contains procs that can compile a static library, dynamic library, or self contained shell directly from exec calls.
I would propose that this tool system (or a new creation by the community in a similar spirit) be included in the library/ section of the tcl core. The provisional name for this tool set would be practcl. A version of this tool could also be provided in tcllib to allow 8.5 and 8.6 based cores to continue to build extensions.
In the new scheme, make packages (in all its forms) would be replaced with a call to "$(TCLSH) ${srcdir}/pkgs/make.tcl build". make packages-install would be replaced by a call to "$(TCLSH) ${srcdir}/pkgs/make.tcl install". For advanced users, these toplevel commands build and install will accept additional arguments. For instance, to install the core distributed packages into the VFS of a kit: "$(TCLSH) ${srcdir}/make.tcl install -destdir ${MyVFS}/lib".
make.tcl would be maintained as part of the core, and provide the top-level control system to build, install, or repackage the core distributed extensions. That script will also provide mechanisms to populate the pkgs file system for developers who build the tcl core from fossil checkouts.
Commands:
basekit
Compile a ZipFs style basekit suitable for the wrap command.
build ?all? ?package? ?package...?
Compile the source code for core distributed packages into binary products (as applicable.) If all is given, an attempt is made to compile all packages under pkgs/. Any other argument is interpreted to be the name of an individual package to be compiled.
install ?-destdir destinationpath?
Install all core distributed packages locally. If -destdir is given, install the packages relative to destinationpath in the same way that "make DESTDIR= destdir" would. If -destdir is not given, or is an empty string, perform an install relative to the exec_prefix in tclConfig.sh
wrap exename vfspath ?dir...?
Generate a self contained executable constructed from the virtual file system amalgamated from vfspath and any other directories given as arguments. This VFS will automatically be populated with the library/ file system from Tcl.
distribution
> Download packages listed in the "packages.tcl" file, unpack their source code in the /pkgs folder, and perform any steps required to prepare those extensions for inclusion in a core snapshot for distribution.
developer
> Download packages listed in the "packages.tcl" file, unpack their source code in the parent folder to the one the core has been unpacked from, and perform any steps required to prepare those extensions to be compiled locally as part of a developer build. This is intended for developers who work from fossil checkouts of the Tcl core.
package-list
> Stream to stdout a list of all of the packages in the packages.tcl file, in a flat machine readable format.
This same mechanism will be adapted for Tk. Tk will be also provide a pkgs/ directory. Its base kits will be based on a modified wish instead of a modified tclsh.
During the build/install/etc phase each directory will be scanned for either a "configure" file or a "prac.tcl" file. Standard TEA extensions will be detected by the presence of a "configure" file. The prac.tcl file is a hint to the build system that the package needs either additional instructions and guidance. The contents of the file are interpreted by the object which is implementing the extension's ambassador to the build system.
If one were to decide to bundle tcllib, and wished to exercise its SAK based installer the prac.tcl file would read:
# Implement the install routine for tcllib # oo::objdefine [self] { method install DEST { set pkg [my define get pkg_name [my define get name]] my unpack set prefix [string trimleft [my <project> define get prefix] /] set srcdir [my define get srcdir] ::practcl::dotclexec [file join $srcdir installer.tcl] \ -pkg-path [file join $DEST $prefix lib $pkg] \ -no-examples -no-html -no-nroff \ -no-wait -no-gui -no-apps } }
Each pkgs/ file system for both Tcl and Tk will also contain a file packages.tcl. This file will be human and machine readable. It contains a description of every core distributed package, where the sources can be found, as well as which fossil tags can be utilized as either development or release with this particular version of the core.
packages.txt contains a series of keywords populating a data structure.
A simple example would by the tclconfig templates from TEA:
EXTENSION tclconfig { tag trunk fossil_url http://core.tcl.tk/tclconfig }
The EXTENSION keyword is intended to take the following arguments:
name key/value-configuration-dict
The list is kept separate from the actual make.tcl so that users can simply steal the list for making "batteries included" distributions. It also allows the package list to remain distinct for each branch of the core.
This document has been placed in the public domain.
[Index] [History] [HTML Format] [Source Format] [LaTeX Format] [Text Format] [XML Format] [*roff Format (experimental)] [RTF Format (experimental)]
TIP AutoGenerator - written by Donal K. Fellows