YAFU 2.0

[bsquared]

Originally Posted by EdH

Could I delete siqs.dat as soon as "SIQS elapsed time" appears in factor.log? Is there any need for siqs.dat to exist after that point? It would be easy for me to have a separate script watching factor.log for that line and invoke a deletion. As long as that wouldn't interfere with normal runs, I'll try it. I'm already deleting siqs.dat prior to each run anyway. This would just do it sooner.

There is no need for the file after that line, no.

[James Heinrich]

Originally Posted by bsquared

There is no need for the file after that line, no.

Would it make any sense for YAFU itself to have an option to auto-delete siqs.dat at the point where it's no longer required? Defaulted off of course, but perhaps easier (or at least more reliable) than depending on external programs to monitor and delete when it's (hopefully) no longer needed (or via manual cleanup).

[bsquared]

Originally Posted by James Heinrich

Would it make any sense for YAFU itself to have an option to auto-delete siqs.dat at the point where it's no longer required? Defaulted off of course, but perhaps easier (or at least more reliable) than depending on external programs to monitor and delete when it's (hopefully) no longer needed (or via manual cleanup).

Yes that does make sense .

There is also the existing "inmem" option. As the name suggests, siqs does all work in RAM and never writes to siqs.dat. Of course if something goes wrong or you want to abort then you have to start over. But for most jobs that's seconds to a few minutes or so. And in EdH's case it would probably solve the looping problem because if no factors are found, the next time around an entirely new matrix will be built.

usage:
-inmem 70

Where 70 (for example) means that all 70 digit inputs or smaller are processed without saving to file.

[James Heinrich]

Originally Posted by bsquared

There is also the existing "inmem" option...
Where 70 (for example) means that all 70 digit inputs or smaller are processed without saving to file.

Ooh, I didn't know about that, new feature in 2.0 I guess (I'm still mostly stuck with 1.34 on my i7-3930K).
It will go nicely with the extension to -pretest feature I requested.

[EdH]

Originally Posted by bsquared

Yes that does make sense .

There is also the existing "inmem" option. As the name suggests, siqs does all work in RAM and never writes to siqs.dat. Of course if something goes wrong or you want to abort then you have to start over. But for most jobs that's seconds to a few minutes or so. And in EdH's case it would probably solve the looping problem because if no factors are found, the next time around an entirely new matrix will be built.

usage:
-inmem 70

Where 70 (for example) means that all 70 digit inputs or smaller are processed without saving to file.

Excellent! I'm changing all my yafu.ini files to inmem=95 (since I'm working at 91 digits right now). I'll let you know if that prevents the machines from looping. . .

Thanks!

[EdH]

Originally Posted by EdH

Excellent! I'm changing all my yafu.ini files to inmem=95 (since I'm working at 91 digits right now). I'll let you know if that prevents the machines from looping. . .

I made it overnight and so far today without any looping, so I'm going to declare this a suitable setting. Additionally, I reset the "inmem" to 70 and wrote a script to factor one of the failed composites repeatedly, trying for a loop. The three machines I ran the script on totaled over 700 iterations without an occurrence. The script is provided below for others to try or comment on.

#!/bin/bash

comp=11539488294330117240202826559598428302679556688248012471560145814831982545559910259149337

count=0
while [ ! -e stopTest ]
  do
    rm siqs.dat 2>/dev/null
    rm factor.log 2>/dev/null
    rm session.log 2>/dev/null
    rm loopTest.log 2>/dev/null
    let count=${count}+1
    dt=$(date)
    echo "Run ${count} started at ${dt:16:11}"
    echo "siqs(${comp})" | ./yafu >>loopTest.log
  done

[EdH]

For ease of updates, etc., I would like to have a single yafu.ini that will work on all my machines (maybe excluding the AMDs). I currently have that for everything but the tune_info. It appears that I can add all the machines' tune_info into a single yafu.ini and the appropriate line will be used for each machine.

Is there a limit to how many tune_info lines I should use? I have several of a particular i7 and they vary in RAM (or other specs). Should I choose the lower or higher set of values, or are they close enough (if even different) that I would notice no difference?

Thanks for all.

[bsquared]

Originally Posted by EdH

For ease of updates, etc., I would like to have a single yafu.ini that will work on all my machines (maybe excluding the AMDs). I currently have that for everything but the tune_info. It appears that I can add all the machines' tune_info into a single yafu.ini and the appropriate line will be used for each machine.

Is there a limit to how many tune_info lines I should use? I have several of a particular i7 and they vary in RAM (or other specs). Should I choose the lower or higher set of values, or are they close enough (if even different) that I would notice no difference?

Thanks for all.

There is no built-in limit. There is nothing computationally intensive about processing them. So, have a ball! If it helps at all the structure of them is:

cpu-string, os-string, siqs-mult, siqs-exp, nfs-mult, nfs-exp, siqs-gnfs-xover, freq

If the line's cpu-string and os-string match the current cpu/os then the rest of the line is parsed and applied, RAM and other system configuration will not enter into it. Really only the siqs-gnfs-xover number is used, the rest is just informational (but parsing expects them to be there - it is not robust to missing info). The mult/exp numbers are constants in an exponential trendline fit to the data measured during tune. The freq is unused, I think tune just assigns it the value '42' for now. Because, reasons.

[EdH]

Originally Posted by bsquared

There is no built-in limit. There is nothing computationally intensive about processing them. So, have a ball! If it helps at all the structure of them is:

cpu-string, os-string, siqs-mult, siqs-exp, nfs-mult, nfs-exp, siqs-gnfs-xover, freq

If the line's cpu-string and os-string match the current cpu/os then the rest of the line is parsed and applied. Really only the siqs-gnfs-xover number is used, the rest is just informational (but parsing expects them to be there - it is not robust to missing info). The mult/exp numbers are constants in an exponential trendline fit to the data measured during tune. The freq is unused, I think tune just assigns it the value '42' for now. Because, reasons.

Excellent! Thanks for the extra info. Does the tune_info take precedence over "xover=100," or do I need to comment (% ) that out?

[bsquared]

Originally Posted by EdH

Excellent! Thanks for the extra info. Does the tune_info take precedence over "xover=100," or do I need to comment (% ) that out?

tune_info should take precedence, but let me know if not.

[EdH]

Originally Posted by bsquared

tune_info should take precedence, but let me know if not.

i have should have checked first, but tune_info is superseded by xover:

fac: using specified qs/gnfs crossover of 100 digits
 fac: using specified qs/snfs crossover of 75 digits

vs.:

fac: using tune info for qs/gnfs crossover

when I comment it out.


Thanks.