Adventures in building and using yafu

[axn]

Colab cheat sheet:
Model: 63 ==> Haswell
Model: 79 ==> Broadwell
Model: 85 ==> Skylake

[EdH]

Originally Posted by axn

Colab cheat sheet:
Model: 63 ==> Haswell
Model: 79 ==> Broadwell
Model: 85 ==> Skylake

Thanks! I've been using:

gcc -march=native -Q --help=target|grep march

[EdH]

Originally Posted by axn

Colab cheat sheet:
Model: 63 ==> Haswell
Model: 79 ==> Broadwell
Model: 85 ==> Skylake

Interesting! Today's Colab wasn't even a Xeon:

  -march=                             znver1
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              2
On-line CPU(s) list: 0,1
Thread(s) per core:  2
Core(s) per socket:  1
Socket(s):           1
NUMA node(s):        1
Vendor ID:           AuthenticAMD
CPU family:          23
Model:               49
Model name:          AMD EPYC 7B12
Stepping:            0
CPU MHz:             2250.000
BogoMIPS:            4500.00
Hypervisor vendor:   KVM
Virtualization type: full
L1d cache:           32K
L1i cache:           32K
L2 cache:            512K
L3 cache:            16384K
NUMA node0 CPU(s):   0,1
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc cpuid extd_apicid tsc_known_freq pni pclmulqdq ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw topoext ssbd ibrs ibpb stibp vmmcall fsgsbase tsc_adjust bmi1 avx2 smep bmi2 rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 clzero xsaveerptr arat npt nrip_save umip rdpid

[James Heinrich]

I tried this today:

/var/www/vhosts/mersenne.ca/yafu/yafu/yafu 'factor(88186028826563457460404188406504565388280836679706598577353981481147908985219332715171968739222503977)'

And it went merrily along for some time (apparently 23 minutes) but then ended with:

...
save 4.619370e-14 -4.5518 557084.16 7.704319e-09 rroots 2
hashtable: 1024 entries, 0.02 MB
elapsed time: 1391.5959 seconds (1414 second deadline); poly select done
nfs: commencing algebraic side lattice sieving over range: 970000 - 990000
sh: ./gnfs-lasieve4I12e: No such file or directory
nfs: could not open output file, possibly bad path to siever
fopen error: No such file or directory
could not open rels0.dat for reading

yafu.ini contains ggnfs_dir=/var/www/vhosts/mersenne.ca/yafu/ggnfs/bin/ and gnfs-lasieve4I12e exists therein.


One thing I just noticed that may be relevant: if I change to /var/www/vhosts/mersenne.ca/yafu/yafu/ first, then start yafu it seems to read yafu.ini:

fac: using tune info for qs/gnfs crossover

but if I start yafu with absolute path but from a different directory, I get this at the top, which I assume means it's not reading yafu.ini?

fac: no tune info: using qs/gnfs crossover of 95 digits
fac: no tune info: using qs/snfs crossover of 75 digits

Trying again, this time starting yafu from the yafu installation directory. And indeed, it runs to completion:

NFS elapsed time = 1892.1189 seconds.
Total factoring time = 2005.8630 seconds
***factors found***
P46 = 3762692755643277024971451604457930207587831987
P56 = 23436946504415561360026241087608631100092994050946766771

Is this expected behavior? Should I have to always change to the yafu installation directory before running it? I would have thought it make more sense that yafu looks for yafu.ini in the directory the executable lives in, no matter where it's called from?

[bsquared]

Originally Posted by James Heinrich

Is this expected behavior? Should I have to always change to the yafu installation directory before running it? I would have thought it make more sense that yafu looks for yafu.ini in the directory the executable lives in, no matter where it's called from?

What you are seeing is the expected behavior: it will look for the .ini in the directory it was run from. Output files also go in the directory it was run from. I guess I pretty much set all I/O to be from where it was run, not stopping to think about the .ini. It would make sense to not have to copy the .ini everywhere.

[EdH]

Actually, I make use of that feature by copying a "working" copy of yafu and yafu.ini into folders where I regularly use it. This way I can update the main YAFU directory and if the latest/greatest doesn't work quite right, my other work isn't interrupted.

[James Heinrich]

I have become confused. When I run this:
'/var/www/vhosts/mersenne.ca/yafu/yafu/yafu' 'factor(123311202266256)' -logfile /dev/null -session /dev/null
from the console, it does what I expect:

fac: factoring 123311202266256
fac: using pretesting plan: normal
fac: using tune info for qs/gnfs crossover
div: primes less than 10000
fmt: 1000000 iterations
Total factoring time = 0.0054 seconds
***factors found***
P1 = 2
P1 = 2
P1 = 2
P1 = 2
P1 = 3
P2 = 23
P12 = 111694929589

But when I run the exact same command as a scheduled task (under the web user) it gives me:

missing variable indicator (@) in input expression
ignoring any input expression: interpreting batchfile lines as input expressions
eof; done processing batchfile

What have I broken?

[bsquared]

Yikes.
I suspect the task is wrapping that command line up into some sort of 'exec' call or using pipes or something, putting yafu into a state where it thinks there is an incoming batchfile.

When you pipe or redirect stuff into yafu, it interprets that as an implicit batchfile with "factor()" wrapped around whatever you are piping/redirecting.

That seems to work fairly well on linux, but I haven't tested much on windows, and I have no real idea what is happening behind the scenes with a scheduled task.

My only suggestion at this point is a workaround. Can you write the lines to factor into a file like worktodo.txt, then have the scheduled task execute ./yafu -batchfile worktodo.txt?

Interesting and irrelevant side-note, when testing this out, I randomly typed in a 16-digit prime number:
echo "1703948751903457" | ./yafu

[EdH]

This isn't just a Windows issue. I've experienced this behavior with bash scripts in linux, as well. I worked around it by going ahead and using a batchfile.

I don't have any examples handy, since it's been a while and I didn't consider bringing the trouble to light. If I run across similar again, I'll take note.

Thinking on this a little more, perhaps related, I have on a couple occasions tried to use YAFU as a big number calculator in bash scripts and (I think) it displayed the same behavior. I may look into this again to see if I can provide more specifics, since it was an odd use attempt. (Kind of like when I tried to use nextprime() on an outrageously large number.)

[bsquared]

This is kinda a dusty back-corner of yafu, unfortunately. I have tried to get it to work "as one would expect" (keeping in mind this may be different for different people, as we saw above) for all combinations of several OS's, terminal types, and use-cases (standard command line, pipes, redirects, yafu's special interactive environment, and -batchfile on the command line). Obviously this only works well enough to pass muster in the more well-worn circumstances.

The main problem right now is I don't know exactly what circumstances are occurring inside a scheduled task.

[James Heinrich]

I should have been clearer, the actual call is coming from a PHP script, called via the backtick operator (aka shell_exec), just like all my calls to external programs.
My test was running the PHP script from the shell as root vs running it as a scheduled task under the user the website runs as (pedantically: clicking 'Run Now' from a new scheduled task in Plesk).
(BTW: I have no idea if this happens on Windows or not, all my questions here are related to the mersenne.ca server which (now) runs CentOS 8)

Originally Posted by bsquared

Interesting and irrelevant side-note, when testing this out, I randomly typed in a 16-digit prime number: echo "1703948751903457" | ./yafu

Your "irrelevant side note" is entirely relevant. If I call it in that manner, echo-piping to yafu rather than passing a 'factor(123)' command, it seems to work both as root and webuser, at least in my test script:

cd /var/www/vhosts/mersenne.ca/yafu/yafu && echo '123311202266256' | ./yafu -logfile /dev/null -session /dev/null
=== Starting work on batchfile expression ===
factor(123311202266256)
=============================================
fac: factoring 123311202266256
fac: using pretesting plan: normal
fac: using tune info for qs/gnfs crossover
div: primes less than 10000
fmt: 1000000 iterations
Total factoring time = 0.0058 seconds
***factors found***
P1 = 2
P1...

So that seems like a simple enough workaround that suits my purposes.

One last question: if I now seem to be reading yafu.ini correctly by cd'ing to the directory first, would this be the correct format to disable (session|factor).log and avoid those parameters?
session="/dev/null"
logfile="/dev/null"