Reading a script gives error, but directly using the script's content works

Questions and problems about using polymake go here.
SimonKing
Posts: 14
Joined: 01 Mar 2017, 12:36

Reading a script gives error, but directly using the script's content works

Postby SimonKing » 23 Mar 2017, 12:49

Hi!

I've put the following into a file `test.pmk`:

Code: Select all

@x=([[12, -2, -3, -5, -8, -13, -21, -34, -55], [0, 1, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0, 1], [0, 0, 0, 0, 0, 0, 0, 1, 0], [0, 0, 0, 0, 0, 0, 1, 0, 0], [0, 0, 0, 0, 0, 1, 0, 0, 0], [0, 0, 0, 0, 1, 0, 0, 0, 0], [0, 0, 0, 1, 0, 0, 0, 0, 0], [0, 0, 1, 0, 0, 0, 0, 0, 0]]);
Then, I tried to read it as a script, which fails:

Code: Select all

polytope > script "test.pmk"; polymake: ERROR: reference to an undeclared variable @x at /home/king/test.pmk line 1.
However, in an interactive session, all works fine:

Code: Select all

polytope > @x=([[12, -2, -3, -5, -8, -13, -21, -34, -55], [0, 1, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0, 1], [0, 0, 0, 0, 0, 0, 0, 1, 0], [0, 0, 0, 0, 0, 0, 1, 0, 0], [0, 0, 0, 0, 0, 1, 0, 0, 0], [0, 0, 0, 0, 1, 0, 0, 0, 0], [0, 0, 0, 1, 0, 0, 0, 0, 0], [0, 0, 1, 0, 0, 0, 0, 0, 0]]); polytope > print $x[0]->[0][1]; -2
My question is: What command must I use in order to read the content of a file so that it is interpreted in the same way as in user interaction? I thought the answer is "script", but I may be mistaken.

Cheers,
Simon

User avatar
gawrilow
Main Author
Posts: 423
Joined: 25 Dec 2010, 17:40

Re: Reading a script gives error, but directly using the script's content works

Postby gawrilow » 23 Mar 2017, 13:22

Every script not located in the specific per-application directory apps/APPNAME/scripts should start with a line

use application "APPNAME";

otherwise no names introduced in the application namespace will be found without full qualification.

Besides this, automatic declaration of variables introduced by assignment is only active in the shell input, as a courtesy to the user; scripts and rulefiles must always declare package-level variables or use lexical variables whenever appropriate. In your case, if this @x is supposed to be referenced later in the interactive shell, you should introduce it as

Code: Select all

declare @x=...;
If it's just a temporary variable for the script, it should be made lexical:

Code: Select all

my @x=...;
I have a faint hope that this is mentioned somewhere in the wiki pages devoted to scripting...

SimonKing
Posts: 14
Joined: 01 Mar 2017, 12:36

Re: Reading a script gives error, but directly using the script's content works

Postby SimonKing » 23 Mar 2017, 13:27

Thank you very much! When I put "declare" into the file then it works (and "declare" also gives no error when used in the command line).

Best regards,
Simon

User avatar
gawrilow
Main Author
Posts: 423
Joined: 25 Dec 2010, 17:40

Re: Reading a script gives error, but directly using the script's content works

Postby gawrilow » 23 Mar 2017, 13:32

Yes, "declare" can safely be used on the command line as well, but only once, when the variable is introduced first time.
For scripts, there is an exception: if a script is executed repeatedly, "declare" are ignored.

SimonKing
Posts: 14
Joined: 01 Mar 2017, 12:36

Re: Reading a script gives error, but directly using the script's content works

Postby SimonKing » 23 Mar 2017, 13:40

Yes, "declare" can safely be used on the command line as well, but only once, when the variable is introduced first time.
For scripts, there is an exception: if a script is executed repeatedly, "declare" are ignored.
Yes, I just noticed.

My question is related with the Sage<->polymake pexpect interface that I created a couple of weeks ago. In Sage's pexpect interfaces, long command lines are usually read from a file, while shorter lines are directly sent via pseudoTty. The latter is (from polymake's perspective) the same as user interaction on the command line.

But if a variable can be declared only once, then it certainly is possible to explicitly un-declare a variable, so that it can later be used again (and of course so that the memory used for the old data is released). How?

Best regards,
Simon

User avatar
gawrilow
Main Author
Posts: 423
Joined: 25 Dec 2010, 17:40

Re: Reading a script gives error, but directly using the script's content works

Postby gawrilow » 23 Mar 2017, 14:04

Freeing the memory is not a big deal, just say

Code: Select all

undef $x;
or

Code: Select all

@x=();
and everything is gone; later you can assign something else to it again. But the name itself stays in the package name table, it's not that easy to get rid of it in a safe manner. Surely, you can destroy the entry in the name table physically:

Code: Select all

delete $Polymake::User::{'x'};
but this has an array of dangerous caveats. First, it would destroy all objects named 'x', also a subroutine x() if there happens to be any around. Second, if you referenced this variable in a script, a compiled form of the script will still be there, for the case you want to rerun it and don't want to pay for repeated re-interpreting. Thus, a frozen copy of this variable will continue to live in the body of the script and be reused with every run despite any assignments you might make to it in the shell or other scripts - until you touch the script file itself, which will enforce its re-interpretation. This might become very confusing.

There is also a special form

Code: Select all

declare local $x;
but as all locals in perl it lasts until the scope is left, thus such a variable is not much different from a lexical. Mostly it's used for TAB completion tests.

SimonKing
Posts: 14
Joined: 01 Mar 2017, 12:36

Re: Reading a script gives error, but directly using the script's content works

Postby SimonKing » 23 Mar 2017, 14:17

Freeing the memory is not a big deal, just say

Code: Select all

undef $x;
or

Code: Select all

@x=();
and everything is gone; later you can assign something else to it again. But the name itself stays in the package name table, ...
... which means I couldn't use "declare" again interactively, but still would have to use "declare" when doing the same assignment using a script. Uff.
...it's not that easy to get rid of it in a safe manner. Surely, you can destroy the entry in the name table physically:

Code: Select all

delete $Polymake::User::{'x'};
but this has an array of dangerous caveats. First, it would destroy all objects named 'x', also a subroutine x() if there happens to be any around.
That doesn't seem to be much of a problem, since it would only concern variable names that Sage has automatically created, and hopefully there is no subroutine in $Polymake::User called `SAGE27`...

Question: Is it enough to do

Code: Select all

delete $Polymake::User::{'x'};
, or do I additionally need

Code: Select all

undef $x;
in order to free the memory?
Second, if you referenced this variable in a script, a compiled form of the script will still be there, for the case you want to rerun it and don't want to pay for repeated re-interpreting. Thus, a frozen copy of this variable will continue to live in the body of the script and be reused with every run despite any assignments you might make to it in the shell or other scripts - until you touch the script file itself, which will enforce its re-interpretation. This might become very confusing.
The scripts I am talking about are created on the fly when Sage attempts to send a long command to polymake. So, even when doing the same command again, one would get the same command but in a new script.

Best regards,
Simon

User avatar
gawrilow
Main Author
Posts: 423
Joined: 25 Dec 2010, 17:40

Re: Reading a script gives error, but directly using the script's content works

Postby gawrilow » 23 Mar 2017, 15:45

... but still would have to use "declare" when doing the same assignment using a script
No, the declaration must happen really only once in the lifetime of any variable. undef or an empty assignment do change the value of a variable but do not affect its lifetime. Any repeated `declare' outside a script would trigger an error, and within a script would be ignored, as I explained - but only once per variable, so if you have declare $x; ... declare $x; in your script, the second line will cause an exception. Neither it's allowed to declare the same variable in two different scripts. They are much like global variables in C/C++.

When you delete a symbol from a package name table, the variable is obviously obliterated (and yes, the value stored in it is automatically destroyed too); after that, you'd need to declare it again if you want to reuse the name.

If you decide to keep variables persistently on the Sage side but destroy them quickly on polymake side, then you are better off with lexical or local declare'd variables. If you are not going to use the full power of the tab completion out of the box, just stick to lexical variables (my $x). They are emptied automatically and don't suffer from any redeclaration issues.

What we need to address somehow is the storage of compiled script bodies. If every new script submitted to polymake has a new unique name, all these bodies will pile up over the time and consume resources in vain. Maybe we should introduce some naming convention for one-shot scripts, e.g. their names should end with ".once" or such. But this still has to be implemented.

User avatar
gawrilow
Main Author
Posts: 423
Joined: 25 Dec 2010, 17:40

Re: Reading a script gives error, but directly using the script's content works

Postby gawrilow » 24 Mar 2017, 00:08

Another idea for your connection: If you are calling polymake from C++ code, you can use the new function simulate_shell_input where you can pass an input string of any length. You won't need to generate any script files at all. The only thing we'd have to add to its implementation is the possibility to pass arguments alongside the string, much like as in call_function(). They will land in a special array @ARGV which you can use in your expression string instead of SAGE27 and such: $ARGV[0] for the first argument and so on. This array is exempt from any redeclaration fuss and will be cleared automatically after the execution of every expression. Would this help you? I could provide this small extension quite quickly.

SimonKing
Posts: 14
Joined: 01 Mar 2017, 12:36

Re: Reading a script gives error, but directly using the script's content works

Postby SimonKing » 24 Mar 2017, 09:50

Another idea for your connection: If you are calling polymake from C++ code, you can use the new function simulate_shell_input where you can pass an input string of any length. You won't need to generate any script files at all.
In a pexpect interface, you do not *simulate* shell input, but you *do* shell input. So, if Sage wants polymake to evaluate cmd="large command string of 1000s of characters", then it may send cmd directly via a pseudo-terminal - but large strings can be problematic in a pseudo-terminal (so, it has to do with limitations in a pseudo-terminal, not in polymake). Therefore, in the case that cmd is large, Sage may also put cmd into tmpfile, and then send the command "please evaluate the content of tmpfile" to polymake via the pseudo-terminal.
The only thing we'd have to add to its implementation is the possibility to pass arguments alongside the string, much like as in call_function(). They will land in a special array @ARGV which you can use in your expression string instead of SAGE27 and such: $ARGV[0] for the first argument and so on. This array is exempt from any redeclaration fuss and will be cleared automatically after the execution of every expression. Would this help you? I could provide this small extension quite quickly.
SAGE27 and so on are not just passed as arguments. They are the variables in which polymake is supposed to store data that Sage wants to be available for some time (in particular, data that it wants to use *after* evaluating the command). Of course, if Sage doesn't need the data any longer, then we want the variable be freed and re-used for new data.

So, I don't see how "simulate_shell_input" could improve the situation for the pexpect interface.

But parallel to the development of a pexpect interface, Vincent Delecroix is creating python bindings (PyPolymake) - I guess such a function would be of use there.

Best regards,
Simon


Return to “Helpdesk”