mersenneforum.org > YAFU Just a Curiosity
 Register FAQ Search Today's Posts Mark Forums Read

 2020-05-31, 19:51 #1 EdH     "Ed Hall" Dec 2009 Adirondack Mtns 10001100111012 Posts Just a Curiosity I recently rebuilt msieve and YAFU to fix a crash and noticed this: Code: 05/31/20 15:36:25 v1.35-beta @ math27, System/Build Info: Using GMP-ECM 7.0.5-dev, Powered by GMP 6.1.2 detected Intel(R) Core(TM) i5-2520M CPU @ 2.50GHz detected L1 = 32768 bytes, L2 = 3145728 bytes, CL = 64 bytes measured cpu frequency ~= 2494.385440 When I ran "tune:" Code: best exponential fit is y = 0.492662 * exp(0.0994914 * x) QS/NFS crossover occurs at 100.3 digits Adding tune_info entry for LINUX64 - Intel(R) Core(TM) i5-2520M CPU @ 2.50GHz Tune info in yafu.ini: Code: tune_info= Intel(R) Core(TM) i5-2520M CPU @ 2.50GHz,LINUX64,2.54497e-05,0.197911,0.492662,0.0994914,100.294,2494.37 When I ran ./yafu via a Perl script: Code: fac: using specified qs/gnfs crossover of 93 digits fac: using specified qs/snfs crossover of 75 digits Edit: I also just noticed this: Code: . . . nfs: commencing polynomial search over range: 17692 - 17942 deadline: 8640000 CPU-seconds per coefficient error generating or reading NFS polynomials nfs: commencing polynomial search over range: 17942 - 18192 deadline: 8640000 CPU-seconds per coefficient coeff 18060 specialq 1 - 946643 other 6855 - 16452 aprogs: 427 entries, 1206 roots hashtable: 1024 entries, 0.02 MB nfs: commencing polynomial search over range: 18192 - 18442 deadline: 8640000 CPU-seconds per coefficient error generating or reading NFS polynomials . . . Last fiddled with by EdH on 2020-05-31 at 19:57
 2020-06-15, 15:49 #2 bsquared     "Ben" Feb 2007 3,617 Posts I'm seeing normal usage of tune info after testing just now: Code: fac: using pretesting plan: normal fac: using tune info for qs/gnfs crossover Make sure that yafu.ini doesn't have lines containing "xover=" or "snfs_xover=" in it. Yafu will also try to verify that the computer/OS info matches between the current CPU and the tune info text. Based on what you posted they should match, so if it doesn't turn out to be a "xover=" line then I'm not sure what's going on. I see the same "error generating or reading NFS polynomials" messages during poly search, but it appears to generate polynomials and complete NFS factorizations just fine. I'll file that as low priority for now. In related news, after the new SIQS updates tune() is now showing a crossover of 106.6 digits for LINUX64! (using a skylake-x.) One should take that number with a large grain of salt given all the assumptions tune() makes, but still, that's higher than I've ever seen it for 64-bit linux ggnfs. I ran the same C100 through both routines and got 641 seconds for SIQS and 681 for NFS, so 106.6 digits is probably a little optimistic toward SIQS. I'll have to re-visit some of the assumptions tune() makes. (I may have shortened poly select time but not reflected that in tune()).
2020-06-15, 18:45   #3
EdH

"Ed Hall"
Dec 2009

33·167 Posts

Quote:
 Originally Posted by bsquared . . . Make sure that yafu.ini doesn't have lines containing "xover=" or "snfs_xover=" in it. Yafu will also try to verify that the computer/OS info matches between the current CPU and the tune info text. Based on what you posted they should match, so if it doesn't turn out to be a "xover=" line then I'm not sure what's going on.
I remember leaving those lines in several of my .ini files. These are probably those machines.

I have noticed the latest .ini version you've created. I haven't delved into the inner portions, but is there a reason that "threads" is commented (%) out, as well as being set to 1?

I've been using your msieve modification to cure segfaults on my GMP 6.2.0 machines. Thanks!

And, thanks for all else, as well.

2020-06-15, 18:51   #4
bsquared

"Ben"
Feb 2007

3,617 Posts

Quote:
 Originally Posted by EdH I have noticed the latest .ini version you've created. I haven't delved into the inner portions, but is there a reason that "threads" is commented (%) out, as well as being set to 1?
In my mind, the .ini file should be used to set *permanent* parameters. Things that you always need (for example the ecm path, ggnfs binary folder, and ecm pretesting preferences). Anything in the .ini file can be overridden on the command line. For me, threads is something I frequently change, depending on what I'm trying to do or what networked system I'm currently running on, so I always set that from the command line. Just my preference: customize the .ini as you see fit. The new .ini is simply meant to document the possible options.

2020-06-15, 18:59   #5
EdH

"Ed Hall"
Dec 2009

119D16 Posts

Quote:
 Originally Posted by bsquared In my mind, the .ini file should be used to set *permanent* parameters. Things that you always need (for example the ecm path, ggnfs binary folder, and ecm pretesting preferences). Anything in the .ini file can be overridden on the command line. For me, threads is something I frequently change, depending on what I'm trying to do or what networked system I'm currently running on, so I always set that from the command line. Just my preference: customize the .ini as you see fit. The new .ini is simply meant to document the possible options.
I just remembered something else to bring up, but will have to wait for another tune session to show an example. I get a repetitive skewed error message with the new .ini file. All seems to run fine afterward, though. This was with and without removing the existing tune data. I suppose it might have to do with the threads edit. I'll post an example later, unless it never shows again.

2020-06-15, 19:04   #6
bsquared

"Ben"
Feb 2007

E2116 Posts

Quote:
 Originally Posted by EdH I just remembered something else to bring up, but will have to wait for another tune session to show an example. I get a repetitive skewed error message with the new .ini file. All seems to run fine afterward, though. This was with and without removing the existing tune data. I suppose it might have to do with the threads edit. I'll post an example later, unless it never shows again.
I saw this too. I think it has to do with blank lines in the .ini. I changed the ReadINI function to ignore these but the writeINI function that is called as part of tune did not get changed and doesn't like them. In mine the blank line were removed, in addition to adding the tune_info line, as an after effect of running tune(). It seemed Mostly Harmless (TM), but I will try to fix at some point anyway.

If you are referring to something else let me know.

2020-06-15, 21:28   #7
EdH

"Ed Hall"
Dec 2009

33×167 Posts

Quote:
 Originally Posted by bsquared I saw this too. I think it has to do with blank lines in the .ini. I changed the ReadINI function to ignore these but the writeINI function that is called as part of tune did not get changed and doesn't like them. In mine the blank line were removed, in addition to adding the tune_info line, as an after effect of running tune(). It seemed Mostly Harmless (TM), but I will try to fix at some point anyway. If you are referring to something else let me know.
I'm pretty sure that's what it was. The blank line was removed in the one I just ran and there was no error - all looked correct.

 2020-06-16, 01:15 #8 EdH     "Ed Hall" Dec 2009 Adirondack Mtns 33·167 Posts Yep, an extra blank line (or two) prior to the previous tune info (and 100.1 digit crossover:) Code: QS/NFS crossover occurs at 100.1 digits Invalid line in yafu.ini, use Keyword=Value pairsSee docfile.txt for valid keywords . . .<38 copies of these lines>. . . Invalid line in yafu.ini, use Keyword=Value pairsSee docfile.txt for valid keywords found OS = LINUX64 and CPU = Intel(R) Xeon(R) Gold 5122 CPU @ 3.60GHz in tune_info field Adding tune_info entry for LINUX64 - Intel(R) Core(TM) i5-2520M CPU @ 2.50GHz Last fiddled with by EdH on 2020-06-16 at 01:17 Reason: repare speelin
 2020-06-16, 01:53 #9 EdH     "Ed Hall" Dec 2009 Adirondack Mtns 450910 Posts I think your larger crossover might explain the results of an experiment I've been conducting: Two machines "virtually" identical. Machine 1: Aliqueit running YAFU for qs, ecm.py for ECM and CADO-NFS for GNFS Machine 2: Aliqueit running option -y (YAFU for all) Both were started on an existing sequence that was around 105 digits. Although Machine 1 seemed to be doing a lot more ECM than Machine 2, it ran away, leaving machine 2 many lines behind. At this time, the sequence had taken a downturn and was shedding digits. I swapped the setups and copied the longer sequence to Machine 2, so they would start even again. This time Machine 1, running YAFU (-y) ran away by over 30 indices, at first! As the digits rose again, Machine 2 caught up with and passed Machine 1, even though it had been 30 indices behind. I will have to do some more comparisons to really conclude anything, but the preliminary shows CADO-NFS to be promising. As to my using CADO-NFS, for my Aliqueit runs, I'm implementing distributed polyselect and sieving across >20 machines of various architecture. I was able to do this with factmsieve.py, too, but CADO-NFS is simpler and now it seems faster, also. I noticed a query as to the possibility of YAFU using CADO-NFS and know this would be quite a bit of work, but from my playing around and other tests, etc., if you ever do consider CADO-NFS, I would suggest still letting Msieve do the LA.
2020-06-16, 02:18   #10
VBCurtis

"Curtis"
Feb 2005
Riverside, CA

5,279 Posts

Quote:
 Originally Posted by EdH ..... if you ever do consider CADO-NFS, I would suggest still letting Msieve do the LA.
I think on small inputs the difference in LA time isn't enough to be worth much programming effort, since it's more complicated to use a hybrid versus cado-as-black-box-factorer. I don't bother to break CADO to invoke msieve for LA below 150 digits, and by that size lots of folks aren't using YAFU anyway. msieve may be 2/3rds the time, but the matrix time for a 130 digit number just isn't that long.

2020-06-16, 11:51   #11
EdH

"Ed Hall"
Dec 2009

33·167 Posts

Quote:
 Originally Posted by VBCurtis I think on small inputs the difference in LA time isn't enough to be worth much programming effort, since it's more complicated to use a hybrid versus cado-as-black-box-factorer. I don't bother to break CADO to invoke msieve for LA below 150 digits, and by that size lots of folks aren't using YAFU anyway. msieve may be 2/3rds the time, but the matrix time for a 130 digit number just isn't that long.
But, if the programming takes care of the hybridization, why not try for the fastest factoring possible? I would think if you already have msieve doing LA and you want to inject CADO into a package for polyselect and sieving, it would be best to just inject that faster part. Why cripple LA, even a little?

 Similar Threads Thread Thread Starter Forum Replies Last Post Gordon PrimeNet 1 2015-07-30 06:49 petrw1 Math 4 2015-07-19 02:33 R.D. Silverman GMP-ECM 4 2012-05-10 20:00 JuanTutors Math 3 2004-10-20 04:37 ET_ Math 7 2004-02-13 17:36

All times are UTC. The time now is 11:38.

Mon May 23 11:38:14 UTC 2022 up 39 days, 9:39, 0 users, load averages: 2.10, 2.00, 1.75