Argument Expansion CommandPeter SpjuthDonal K. FellowsAndreas Leitgeb$Revision: 1.13 $
This TIP proposes to add a command that can perform argument expansion in a safe and efficient manner.
Many commands take a variable number of arguments and often you find yourself with those arguments in a list. This list must then be expanded into individual arguments to the command. This is currently done with eval:
ZXZhbCBkZXN0cm95IFt3aW5mbyBjaGlsZHJlbiAuXQ==
This is a bit obscure and also very error prone when the command becomes more complex. It is also inefficient and not object safe, why a command specialised in doing this would be better.
There have been suggestions of introducing some new syntax to Tcl to handle argument expansion. That is a big and controversial step, and not anything this TIP wants to meddle in. A command can improve every point where eval has shortcomings and thus give a good result with less means. It can also serve as a bridge to a future global syntax.
Such a command can be done in several ways and below the choice in this TIP's specification is defended.
As examples three statements are used which will be repeated for different alternatives. This is the eval version:
ZXZhbCBkZXN0cm95IFt3aW5mbyBjaGlsZHJlbiAuXQ==ZXZhbCBidXR0b24gLmIgJHN0ZGFyZ3MgLXRleHQgXCRteXRleHQgLWJkICRib3JkZXI=ZXZhbCBleGVjIFwkcHJvZyAkb3B0czEgW2dldE1vcmVvcHRzXSBcJGZpbGUxIFwkZmlsZTI=
The eval version would be even more complex if the lists that are to be expanded are not known to be pure. To be really safe the last would be:
ZXZhbCBleGVjIFwkcHJvZyBbbHJhbmdlICRvcHRzMSAwIGVuZF0gW2xyYW5nZSBbZ2V0TW9yZW9wdHNdIDAgZW5kXSBcJGZpbGUxIFwkZmlsZTI=
With the proposed command they become:
ZXhwYW5kIHsgZGVzdHJveSBgW3dpbmZvIGNoaWxkcmVuIC5dIH0=ZXhwYW5kIHsgYnV0dG9uIC5iIGAkc3RkYXJncyAtdGV4dCAkbXl0ZXh0IC1iZCAkYm9yZGVyIH0=ZXhwYW5kIHsgZXhlYyAkcHJvZyBgJG9wdHMxIGBbZ2V0TW9yZW9wdHNdICRmaWxlMSAkZmlsZTIgfQ==
An alternative to having a local syntax is to point at the arguments that should be expanded, either by index:
ZXhwYW5kIHtlbmR9IGRlc3Ryb3kgW3dpbmZvIGNoaWxkcmVuIC5dZXhwYW5kIHsyfSBidXR0b24gLmIgJHN0ZGFyZ3MgLXRleHQgJG15dGV4dCAtYmQgJGJvcmRlcg==ZXhwYW5kIHsyIDN9IGV4ZWMgJHByb2cgJG9wdHMxIFtnZXRNb3Jlb3B0c10gJGZpbGUxICRmaWxlMg==
Or by some flag mechanism:
ZXhwYW5kIGRlc3Ryb3kgKyBbd2luZm8gY2hpbGRyZW4gLl0=ZXhwYW5kIGJ1dHRvbiAuYiArICRzdGRhcmdzIC10ZXh0IC0gJG15dGV4dCAtYmQgJGJvcmRlcg==ZXhwYW5kIGV4ZWMgLSAkcHJvZyArICRvcHRzMSArIFtnZXRNb3Jlb3B0c10gLSAkZmlsZTEgLSAkZmlsZTI=
Those lack in writability/readability/maintainability in a disturbing manner.
For the choice of local syntax one goal is that it should not violate Tcl's rules, which simplifies implementation since Tcl's parser can do the job.
Any char that fulfils that could be used but the choice fell on ` for forward compatibility reasons. See below.
An alternative syntax could be using enclosing `` or some other enclosing construct like:
ZXhwYW5kIHsgZGVzdHJveSA8W3dpbmZvIGNoaWxkcmVuIC5dPiB9ZXhwYW5kIHsgYnV0dG9uIC5iIDwkc3RkYXJncz4gLXRleHQgJG15dGV4dCAtYmQgJGJvcmRlciB9ZXhwYW5kIHsgZXhlYyAkcHJvZyA8JG9wdHMxPiA8W2dldE1vcmVvcHRzXT4gJGZpbGUxICRmaWxlMiB9
Paired characters are good for delimiting things. Here is the beginning; here is the end. But this is not about a new way to delimit things. It is about indicating a boolean choice: expand or do not expand a word into multiple words. Whatever character is chosen to be that indicator, it should be a single, leading one. No pairs.
In the specification a restrictive rule was chosen that makes it an error to use ` in a way that do not fit. This is to make it easier to change things in the future should ideas come up for new features. E.g., should this become a global syntax in Tcl 9.0 it can be chosen a bit differently and be backward compatible with the expand command.
A new command "expand" is added. It takes one argument, which contains a Tcl script consisting of one command. The script may contain comments but only one command is permitted.
The command is processed in the following manner:
Parse into words according to Tcl's standard rules.Any word starting with ` must be followed by a single variable or command substitution. The word is remembered and the ` is removed.Perform Tcl's normal execution steps on the new line up to the point where the command should have been called.Expand the arguments that should be expanded.Execute the command.
The return value of expand is the return value of the command.
Note 1: A word should really start with ` to trigger expansion which means that words like these are not expanded:
Y21kICJgJHRlbXAiIFxgW3NvbWV0aGluZ10=
Note 2: Expansion is only performed with words like:
Y21kIGAkdmFyIGBbc29tZWNtZCAkYXJnXSBgJGFycihbY21kICRhcmddKQ==
Words like these are a syntax error:
Y21kIGB3b3JkYCBgJHgsJHkgYFtmb29deHlbYXBhXQ==
One aspect of choosing a syntax here is to think about the future. Should there later be a wish for a global syntax for argument expansion it would be nice if it were the same as the one chosen in the expand command. If an agreement can be made for what may be acceptable in the future, this should affect the specification in this TIP.
If a single character like ` is chosen for a global expand syntax it means a backwards compatibility break. So, what chars are likely to be used by people and thus causing problems or confusion when backwards compatibility is broken?
Some food for thought about different chars:
XyAgICAgIyBXb3JkIGNoYXI=OiAgICAgIyBHZXRzIHVnbHkgd2l0aCBuYW1lc3BhY2UgcXVhbGlmaWVyczogIDokOjp2YXI=
ISAgIGlmICEkdmFyIHsuLi59KiAgIHN0cmluZyBtYXRjaCAqJHN1ZmZpeCAkbGluZQ==XiAgIHJlZ2V4cCBeJHByZWZpeCAkbGluZQ==fiAgIGNkIH4kdXNlcg==fCAgIG9wZW4gfCRwcm9nLiAgIGJ1dHRvbiAuJHcgOyBnbG9iIC1ub2NvbXBsYWluIC4kc3RyPSAgIHdtIGdlb21ldHJ5IC5lID0kZ2VvQCAgIC54IGNvbmYgLWJpdG1hcCBAJGJtcCAtY3Vyc29yIEAkY3VyPCAgIGJpbmQgLiA8JGxlZnQ+ICA7IHNldCBodG1sIDwkdGFnPg==
KCAgIGV4cHIgKCRhICsgJGIpICogJGMgOyMgQ29uZnVzZXMgcGFyZW4tbWF0Y2hpbmc=KSAgICAgIyBPZGQgZW5vdWdoIGFzIG9wZW5pbmcsIGJ1dCB3b3VsZCBjb25mdXNlIGFueSBwYXJlbi1tYXRjaGluZw==
KyAgIGV4cHIgJGEgKyRiICA7IyBTYW1lIGZvciBhbnkgb3BlcmF0b3I=LSAgIA==JQ==JiAgIA==Pw==LyAgIG9wZW4gLyRwYXRoICAg
JyAgICAgIyBNYWtlcyBtb3JlIHNlbnNlIGFzIGVuY2xvc2luZz8=YCAgICAgIyBNYWtlcyBtb3JlIHNlbnNlIGFzIGVuY2xvc2luZz8=PiAgIGV4ZWMgZm9vYmFyID4vc29tZS9maWxlLCAgIGFwcGVuZCByZWNpcGllbnRzICxbam9pbiAkaGVhZGVyKCRjY0wpICxd
e30gIGNvbXBsZXRlbHkgZm9yd2FyZC1jb21wYXRpYmxlLCBhcyB7fSBjdXJyZW50bHkgY2Fubm90IGJlICAgIHRyYWlsZWQgYnkgYW55dGhpbmcgYnV0IHdoaXRlc3BhY2UuIChUaGlzIHdvdWxkIGxpbWl0IHRoZQ==ICAgIG9yaWdpbmFsbHkgcHJvcG9zZWQgZ2xvYmFsIHN5bnRheCBjaGFuZ2UgdG8gdGhlIGFyZ3VtZW50ICAgIG9mIHRoZSBleHBhbmQgY29tbWFuZCk=
Example usage of those that seem reasonable:
ZXhwYW5kIHsgZXhlYyAkcHJvZyAnJG9wdHMxICdbZ2V0TW9yZW9wdHNdICRmaWxlMSAkZmlsZTIgfQ==ZXhwYW5kIHsgZXhlYyAkcHJvZyBgJG9wdHMxIGBbZ2V0TW9yZW9wdHNdICRmaWxlMSAkZmlsZTIgfQ==ZXhwYW5kIHsgZXhlYyAkcHJvZyAsJG9wdHMxICxbZ2V0TW9yZW9wdHNdICRmaWxlMSAkZmlsZTIgfQ==
For comparison, the syntax that has been proposed earlier that would not break backwards compatibility:
ZXhwYW5kIHsgZXhlYyAkcHJvZyB7fSRvcHRzMSB7fVtnZXRNb3Jlb3B0c10gJGZpbGUxICRmaWxlMiB9ZXhwYW5kIHsgZXhlYyAkcHJvZyB7ZXhwYW5kfSRvcHRzMSB7ZXhwYW5kfVtnZXRNb3Jlb3B0c10gJGZpbGUxICRmaWxlMiB9
When first issued the TIP caused some discussion on c.l.t. Until a summary is made, here is the thread:
Another thread about it:
This document has been placed in the public domain.