mersenneforum.org

mersenneforum.org (https://www.mersenneforum.org/index.php)
-   YAFU (https://www.mersenneforum.org/forumdisplay.php?f=96)
-   -   YAFU 2.0 (https://www.mersenneforum.org/showthread.php?t=26681)

kotenok2000 2022-08-10 14:07

Why does yafu remove entry from batchfile when i manually interrupt yafu wuth ctrl+c?

bsquared 2022-08-10 14:39

[QUOTE=kotenok2000;611135]Why does yafu remove entry from batchfile when i manually interrupt yafu wuth ctrl+c?[/QUOTE]

It removes lines after they are completed... it doesn't check that they were completed successfully.

I've always been on the fence about whether it should remove lines at all... on the one hand it's nice to know where you are in the process if something gets interrupted (e.g., power outage, in which case the line wouldn't be removed). On the other there are situations like this that are difficult to handle.

Since I don't want to deal with difficult situations I will probably just get rid of the removing-lines-as-they-finish feature unless there are strong opinions about keeping it.

LaurV 2022-08-10 15:28

1 Attachment(s)
[QUOTE=bsquared;611129]That said, I'm not sure how you are getting both compressed and uncompressed relations in a single file.[/QUOTE]
I don't. They are all compressed. But the machines run other things too, and in a not very stable environment, and the relations files get HUUGE. Sometimes a chunk inside of such a file is damaged, and yafu/gzip can not go behind of it. In a not-compressed file the things would be as easy as deleting few lines of text, with a text editor that can open such large files, or sometimes, not even that, because yafu will just skip lines with the bad relations. Also, when the computer crashes and rels0, rels1, ... rels_n (which are always uncompressed) remain hanging, on a not-compressed environment I could concatenate all these files to the rels.dat with a simple DOS copy command, and salvage the relations, before resuming the sieving. That is why my pledge for a non-compressed dat file. With a compressed dat file, I can not do any of those, and editing the file when things crash is more tedious (as explained in the previous post, something the "normal" user, who doesn't eat hex and binary for breakfast as we eat, can not do). The only disadvantage of an uncompressed dat file would be the larger size, and a bit longer time to read the relations when filtering (the relations are read from disk, as opposite to the zipped file which is decompressed in ram, faster), but nowadays everybody has large disks with lots of free space, and the additional 5-10 minutes, 3 or 4 times per day, won't be an issue.

I am using executable yafu, not compiled by myself. Which compressed the relations. I remember now that you told me once about recompiling msieve, but that I never did...

[ATTACH]27182[/ATTACH]

James Heinrich 2022-08-10 15:29

[QUOTE=bsquared;611137]I will probably just get rid of the removing-lines-as-they-finish feature unless there are strong opinions about keeping it.[/QUOTE]I have a strong preference to keep it for my own use. If you want to disable that by default then please leave the code in there behind a [c]--auto-remove-batch-lines[/c] or similar switch, please.

chris2be8 2022-08-10 15:59

To recover data from a damaged .gz file try gzrecover. My script to recover from rubbish in relations from NFS@Home uses:
[code]
gzrecover -p $GZ | remdups 1250 -v >$DAT
[/code]

$GZ is file to process.
$DAT is where to put the recovered relations.

James Heinrich 2022-08-10 21:22

Just ran into something in today's v2.09 that I haven't seen before:[code]=== Starting work on batchfile expression ===
factor(9893910732172879371097056787104792163954805874908431656489378038530926857)
=============================================
fac: factoring 9893910732172879371097056787104792163954805874908431656489378038530926857
fac: using pretesting plan: deep
fac: using specified qs/gnfs crossover of 100 digits
fac: using specified qs/snfs crossover of 75 digits
div: primes less than 10000
fmt: 1000000 iterations
rho: x^2 + 3, starting 1000 iterations on C73
rho: x^2 + 2, starting 1000 iterations on C73
rho: x^2 + 1, starting 1000 iterations on C73
pm1: starting B1 = 150K, B2 = gmp-ecm default on C73
ecm: 30/30 curves on C73, B1=2k, B2=gmp-ecm default
ecm: 74/74 curves on C73, B1=11k, B2=gmp-ecm default
fopen error: Too many open files
could not open factor.log for appending
ecm: ECM executable does not exist at C:\Users\User\Desktop\yafu2\ecm\ecm.exe
ecm: using internal single threaded ECM...
ecm: 74/74 curves on C73, B1=11k, B2=gmp-ecm default
fopen error: Too many open files
could not open factor.log for appending
ecm: ECM executable does not exist at C:\Users\User\Desktop\yafu2\ecm\ecm.exe
ecm: using internal single threaded ECM...
ecm: 74/74 curves on C73, B1=11k, B2=gmp-ecm default
fopen error: Too many open files
could not open factor.log for appending
ecm: ECM executable does not exist at C:\Users\User\Desktop\yafu2\ecm\ecm.exe
ecm: using internal single threaded ECM...
ecm: 74/74 curves on C73, B1=11k, B2=gmp-ecm default
fopen error: Too many open files
could not open factor.log for appending
ecm: ECM executable does not exist at C:\Users\User\Desktop\yafu2\ecm\ecm.exe
ecm: using internal single threaded ECM...
ecm: 74/74 curves on C73, B1=11k, B2=gmp-ecm default
fopen error: Too many open files
could not open factor.log for appending
ecm: ECM executable does not exist at C:\Users\User\Desktop\yafu2\ecm\ecm.exe
ecm: using internal single threaded ECM...
ecm: 74/74 curves on C73, B1=11k, B2=gmp-ecm default
fopen error: Too many open files
could not open factor.log for appending
ecm: ECM executable does not exist at C:\Users\User\Desktop\yafu2\ecm\ecm.exe
ecm: using internal single threaded ECM...
ecm: 74/74 curves on C73, B1=11k, B2=gmp-ecm default
fopen error: Too many open files
<loop ad infinitum>[/code]and it kept looping until I cancelled it.
(The ECM executable does exist there, I think it just couldn't open it due to too many open files. And of course [c]factor.log[/c] does exist.)
I would suggest that "too many open files" should be a fatal error and YAFU should simply exit at that point rather than looping.
The screen output would just keep overwriting the [c]ecm: xx/74 curves on C73[/c] line over and over, however factor.log seemed to contain a full/normal run log for repeated factoring:[code]08/10/22 17:07:55, ****************************
08/10/22 17:07:55, Starting factorization of 9893910732172879371097056787104792163954805874908431656489378038530926857
08/10/22 17:07:55, using pretesting plan: deep
08/10/22 17:07:55, using specified qs/gnfs crossover of 100 digits
08/10/22 17:07:55, using specified qs/snfs crossover of 75 digits
08/10/22 17:07:55, ****************************
08/10/22 17:07:55, rho: x^2 + 3, starting 1000 iterations on C73
08/10/22 17:07:55, rho: x^2 + 2, starting 1000 iterations on C73
08/10/22 17:07:55, rho: x^2 + 1, starting 1000 iterations on C73
08/10/22 17:07:55, pm1: starting B1 = 150K, B2 = gmp-ecm default on C73
08/10/22 17:07:55, current ECM pretesting depth: 0.000000
08/10/22 17:07:55, scheduled 30 curves at B1=2000 toward target pretesting depth of 24.333333
08/10/22 17:07:55, Finished 30 curves using GMP-ECM method on C73 input, B1=2k, B2=gmp-ecm default
08/10/22 17:07:55, current ECM pretesting depth: 15.177725
08/10/22 17:07:55, scheduled 74 curves at B1=11000 toward target pretesting depth of 24.333333
08/10/22 17:07:56, Finished 74 curves using GMP-ECM method on C73 input, B1=11k, B2=gmp-ecm default
08/10/22 17:07:56, current ECM pretesting depth: 20.242996
08/10/22 17:07:56, scheduled 176 curves at B1=50000 toward target pretesting depth of 24.333333
08/10/22 17:08:00, Finished 176 curves using GMP-ECM method on C73 input, B1=50k, B2=gmp-ecm default
08/10/22 17:08:00, final ECM pretested depth: 24.355146
08/10/22 17:08:00, scheduler: switching to sieve method
08/10/22 17:08:00, starting SIQS on c73: 9893910732172879371097056787104792163954805874908431656489378038530926857
08/10/22 17:08:00, random seed: 10805957936155471635
08/10/22 17:08:00, ==== sieve params ====
08/10/22 17:08:00, n = 76 digits, 251 bits
08/10/22 17:08:00, factor base: 21952 primes (max prime = 527987)
08/10/22 17:08:00, single large prime cutoff: 44878895 (85 * pmax)
08/10/22 17:08:00, double large prime range from 278770272169 to 24607855715335
08/10/22 17:08:00, DLP MFB = 1.75
08/10/22 17:08:00, allocating 6 large prime slices of factor base
08/10/22 17:08:00, buckets hold 2048 elements
08/10/22 17:08:00, large prime hashtables have 491520 bytes
08/10/22 17:08:00, using AVX2 enabled 32k sieve core
08/10/22 17:08:00, sieve interval: 5 blocks of size 32768
08/10/22 17:08:00, polynomial A has ~ 9 factors
08/10/22 17:08:00, using multiplier of 233
08/10/22 17:08:00, using multiplier of 233
08/10/22 17:08:00, using Q2(x) polynomials for kN mod 8 = 1
08/10/22 17:08:00, using SPV correction of 21 bits, starting at offset 34
08/10/22 17:08:00, trial factoring cutoff at 81 bits
08/10/22 17:08:00, ==== sieving started ( 4 threads) ====
08/10/22 17:08:08, trial division touched 2830496 sieve locations out of 15199764480
08/10/22 17:08:08, total reports = 2830496, total surviving reports = 1092798
08/10/22 17:08:08, total blocks sieved = 463860, avg surviving reports per block = 2.36
08/10/22 17:08:08, dlp-ecm: 0 failures, 54248 attempts, 572102 outside range, 359414 prp, 53453 useful
08/10/22 17:08:08, 22363 relations found: 9439 full + 12924 from 151048 partial, using 46386 polys (333 A polys)
08/10/22 17:08:08, on average, sieving found 3.46 rels/poly and 20068.37 rels/sec
08/10/22 17:08:08, trial division touched 2830496 sieve locations out of 15199764480
08/10/22 17:08:08, ==== post processing stage (msieve-1.38) ====
08/10/22 17:08:08, QS elapsed time = 7.9990 seconds.
08/10/22 17:08:08, begin singleton removal with 160487 relations
08/10/22 17:08:08, reduce to 38458 relations in 7 passes
08/10/22 17:08:08, recovered 38458 relations
08/10/22 17:08:08, recovered 26065 polynomials
08/10/22 17:08:08, attempting to build 22363 cycles
08/10/22 17:08:08, found 22363 cycles from 38458 relations in 3 passes
08/10/22 17:08:08, distribution of cycle lengths:
08/10/22 17:08:08, length 1 : 9439
08/10/22 17:08:08, length 2 : 7879
08/10/22 17:08:08, length 3 : 3353
08/10/22 17:08:08, length 4 : 1161
08/10/22 17:08:08, length 5 : 383
08/10/22 17:08:08, length 6 : 106
08/10/22 17:08:08, length 7 : 28
08/10/22 17:08:08, length 9+: 14
08/10/22 17:08:08, largest cycle: 10 relations
08/10/22 17:08:08, matrix is 21952 x 22363 (3.6 MB) with weight 758624 (33.92/col)
08/10/22 17:08:08, sparse part has weight 758624 (33.92/col)
08/10/22 17:08:08, filtering completed in 3 passes
08/10/22 17:08:08, matrix is 17396 x 17460 (2.9 MB) with weight 619158 (35.46/col)
08/10/22 17:08:08, sparse part has weight 619158 (35.46/col)
08/10/22 17:08:08, saving the first 48 matrix rows for later
08/10/22 17:08:09, matrix is 17348 x 17460 (2.4 MB) with weight 510149 (29.22/col)
08/10/22 17:08:09, sparse part has weight 461081 (26.41/col)
08/10/22 17:08:09, matrix includes 64 packed rows
08/10/22 17:08:09, commencing Lanczos iteration
08/10/22 17:08:09, memory use: 3.1 MB
08/10/22 17:08:10, lanczos halted after 276 iterations (dim = 17348)
08/10/22 17:08:10, recovered 17 nontrivial dependencies
08/10/22 17:08:10, prp40 = 1360456298247937862016941462358767177779
08/10/22 17:08:10, prp34 = 7272494342460497523675029892412883
08/10/22 17:08:10, Lanczos elapsed time = 2.0160 seconds.
08/10/22 17:08:10, Sqrt elapsed time = 0.0110 seconds.
08/10/22 17:08:10, SIQS elapsed time = 10.0265 seconds.
08/10/22 17:08:10,
08/10/22 17:08:10,
08/10/22 17:08:10, Total factoring time = 15.4097 seconds[/code]And factor.json also contains multiple entries, however I found it unexpected that it seems to alternate the order of the found factors:[code]{ "input-expression":"factor(9893910732172879371097056787104792163954805874908431656489378038530926857)", "input-decimal":"9893910732172879371097056787104792163954805874908431656489378038530926857", "input-argument-string":"factor(@) -batchfile random_composites.txt ", "factors-prime":["1360456298247937862016941462358767177779","7272494342460497523675029892412883"], "pm1-curves" : {"150000":1}, "ecm-curves" : {"2000":30,"11000":74,"50000":176}, "ecm-levels" : {"t15":32.31,"t20":9.42,"t25":0.87,"t30":0.06}, "runtime" : {"total":14.9000, "ecm":3.7169, "pm1":0.0352, "pp1":0.0000, "siqs":10.3100, "nfs-total":0.0000, "nfs-poly":0.0000, "nfs-sieve":0.0000, "nfs-filter":0.0000, "nfs-la":0.0000, "nfs-sqrt":0.0000}, "time-start" : "2022-08-10 17:06:52", "time-end" : "2022-08-10 17:07:07", "info":{"compiler":"MSVC 1931","ECM-version":"7.0.4","MPIR-version":"3.0.0","yafu-version":"2.09"} }
{ "input-expression":"factor(9893910732172879371097056787104792163954805874908431656489378038530926857)", "input-decimal":"9893910732172879371097056787104792163954805874908431656489378038530926857", "input-argument-string":"factor(@) -batchfile random_composites.txt ", "factors-prime":["7272494342460497523675029892412883","1360456298247937862016941462358767177779"], "pm1-curves" : {"150000":1}, "ecm-curves" : {"2000":30,"11000":74,"50000":176}, "ecm-levels" : {"t15":32.31,"t20":9.42,"t25":0.87,"t30":0.06}, "runtime" : {"total":14.2596, "ecm":4.0419, "pm1":0.0347, "pp1":0.0000, "siqs":9.3449, "nfs-total":0.0000, "nfs-poly":0.0000, "nfs-sieve":0.0000, "nfs-filter":0.0000, "nfs-la":0.0000, "nfs-sqrt":0.0000}, "time-start" : "2022-08-10 17:07:07", "time-end" : "2022-08-10 17:07:21", "info":{"compiler":"MSVC 1931","ECM-version":"7.0.4","MPIR-version":"3.0.0","yafu-version":"2.09"} }
{ "input-expression":"factor(9893910732172879371097056787104792163954805874908431656489378038530926857)", "input-decimal":"9893910732172879371097056787104792163954805874908431656489378038530926857", "input-argument-string":"factor(@) -batchfile random_composites.txt ", "factors-prime":["1360456298247937862016941462358767177779","7272494342460497523675029892412883"], "pm1-curves" : {"150000":1}, "ecm-curves" : {"2000":30,"11000":74,"50000":176}, "ecm-levels" : {"t15":32.31,"t20":9.42,"t25":0.87,"t30":0.06}, "runtime" : {"total":16.6600, "ecm":4.5211, "pm1":0.0445, "pp1":0.0000, "siqs":10.7723, "nfs-total":0.0000, "nfs-poly":0.0000, "nfs-sieve":0.0000, "nfs-filter":0.0000, "nfs-la":0.0000, "nfs-sqrt":0.0000}, "time-start" : "2022-08-10 17:07:21", "time-end" : "2022-08-10 17:07:38", "info":{"compiler":"MSVC 1931","ECM-version":"7.0.4","MPIR-version":"3.0.0","yafu-version":"2.09"} }
{ "input-expression":"factor(9893910732172879371097056787104792163954805874908431656489378038530926857)", "input-decimal":"9893910732172879371097056787104792163954805874908431656489378038530926857", "input-argument-string":"factor(@) -batchfile random_composites.txt ", "factors-prime":["7272494342460497523675029892412883","1360456298247937862016941462358767177779"], "pm1-curves" : {"150000":1}, "ecm-curves" : {"2000":30,"11000":74,"50000":176}, "ecm-levels" : {"t15":32.31,"t20":9.42,"t25":0.87,"t30":0.06}, "runtime" : {"total":16.7173, "ecm":5.0581, "pm1":0.0360, "pp1":0.0000, "siqs":10.4897, "nfs-total":0.0000, "nfs-poly":0.0000, "nfs-sieve":0.0000, "nfs-filter":0.0000, "nfs-la":0.0000, "nfs-sqrt":0.0000}, "time-start" : "2022-08-10 17:07:38", "time-end" : "2022-08-10 17:07:55", "info":{"compiler":"MSVC 1931","ECM-version":"7.0.4","MPIR-version":"3.0.0","yafu-version":"2.09"} }
{ "input-expression":"factor(9893910732172879371097056787104792163954805874908431656489378038530926857)", "input-decimal":"9893910732172879371097056787104792163954805874908431656489378038530926857", "input-argument-string":"factor(@) -batchfile random_composites.txt ", "factors-prime":["1360456298247937862016941462358767177779","7272494342460497523675029892412883"], "pm1-curves" : {"150000":1}, "ecm-curves" : {"2000":30,"11000":74,"50000":176}, "ecm-levels" : {"t15":32.31,"t20":9.42,"t25":0.87,"t30":0.06}, "runtime" : {"total":15.4097, "ecm":4.5138, "pm1":0.0402, "pp1":0.0000, "siqs":10.0265, "nfs-total":0.0000, "nfs-poly":0.0000, "nfs-sieve":0.0000, "nfs-filter":0.0000, "nfs-la":0.0000, "nfs-sqrt":0.0000}, "time-start" : "2022-08-10 17:07:55", "time-end" : "2022-08-10 17:08:10", "info":{"compiler":"MSVC 1931","ECM-version":"7.0.4","MPIR-version":"3.0.0","yafu-version":"2.09"} }
{ "input-expression":"factor(9893910732172879371097056787104792163954805874908431656489378038530926857)", "input-decimal":"9893910732172879371097056787104792163954805874908431656489378038530926857", "input-argument-string":"factor(@) -batchfile random_composites.txt ", "factors-prime":["7272494342460497523675029892412883","1360456298247937862016941462358767177779"], "pm1-curves" : {"150000":1}, "ecm-curves" : {"2000":30,"11000":74,"50000":176}, "ecm-levels" : {"t15":32.31,"t20":9.42,"t25":0.87,"t30":0.06}, "runtime" : {"total":14.0025, "ecm":3.7470, "pm1":0.0336, "pp1":0.0000, "siqs":9.3725, "nfs-total":0.0000, "nfs-poly":0.0000, "nfs-sieve":0.0000, "nfs-filter":0.0000, "nfs-la":0.0000, "nfs-sqrt":0.0000}, "time-start" : "2022-08-10 17:08:10", "time-end" : "2022-08-10 17:08:24", "info":{"compiler":"MSVC 1931","ECM-version":"7.0.4","MPIR-version":"3.0.0","yafu-version":"2.09"} }[/code]Of course the too many open files might be Windows misbehaving rather than YAFU (some update was trying to apply itself in the background at the time I noticed the YAFU problem by chance), but if the error is detected it should exit rather than loop.

bsquared 2022-08-10 21:50

How many entries does your batchfile have? I'm wondering if something isn't getting closed properly when looping over batchfile entries.

James Heinrich 2022-08-10 21:55

[QUOTE=bsquared;611176]How many entries does your batchfile have? I'm wondering if something isn't getting closed properly when looping over batchfile entries.[/QUOTE]I process batches of 100 at a time, submit results, fetch new, re-launch YAFU against the next batch.

I've done this successfully for long periods with v1.34.5 but only just started this on YAFU 2.09 today because the [c]-batchfile[/c] option was broken until this morning.

I'm not fully convinced that it's a YAFU-error rather than a Windows-error, just that YAFU's handling of the error condition should be better.

bsquared 2022-08-10 22:20

[QUOTE=James Heinrich;611177]I process batches of 100 at a time, submit results, fetch new, re-launch YAFU against the next batch.

I've done this successfully for long periods with v1.34.5 but only just started this on YAFU 2.09 today because the [c]-batchfile[/c] option was broken until this morning.

I'm not fully convinced that it's a YAFU-error rather than a Windows-error, just that YAFU's handling of the error condition should be better.[/QUOTE]

I've just done some tests with larger batch files and I'm having problems too. There is some kind of memory or file descriptor leak somewhere that I'll have to track down. Sorry about that! Smaller batches seem to work... which of course doesn't fix the problem but might be a work around for now.

James Heinrich 2022-08-10 23:35

[QUOTE=bsquared;611181]Smaller batches seem to work...[/QUOTE]I've always done batch of 100. I've completed 14x 100 since restarting after posting my bug report and it's been going fine (so far).

bsquared 2022-08-11 14:38

After some longer tests, I've checked in a few fixes and added some missing traps for file open errors. The .exe is updated. Long batchfiles are working fine for me now too.

James Heinrich 2022-08-11 15:07

It had been running fine with 100-batches for about 16h, but I've switched to the new build.

If I may repeat a request in the JSON output, would it be possible to add a [c]sum[/c] key in addition the the [c]tXX[/c] keys? I know we can calculate it ourselves, but it's easier if it's just there.

Also, perhaps for more-compact output the not-used values could be omitted -- for example if P+1, NFS, etc wasn't run there's no need to output the zero values: [c]"pp1":0.0000[/c], [c]"nfs-total":0.0000, "nfs-poly":0.0000, "nfs-sieve":0.0000, "nfs-filter":0.0000, "nfs-la":0.0000, "nfs-sqrt":0.0000[/c]. That's the beauty of JSON -- fields can be added or omitted without breaking the output.

EdH 2022-08-12 14:53

Quick question: Can YAFU pass on GPU use for Msieve calls. I'm not seeing it in the docs. I thought I had in the past.

BudgieJane 2022-08-14 22:00

[QUOTE=bsquared;611133]Checked in new update and new windows exe:

+ by default json output is now all on one line, -jsonpretty option to pretty print like before (each category on separate line)[/QUOTE]

How do I put that jsonpretty option into yafu.ini? When I just put the word on a line by itself (as with similar options in the sample yafu.ini) it seems to be ignored. I've checked and I haven't misspelt it.

Stargate38 2022-08-16 17:17

Try "jsonpretty=1". I've noticed that for a given flag that putting the flag name, followed by "=1" (i.e. v=1) works.

BudgieJane 2022-08-16 22:50

[QUOTE=Stargate38;611580]Try "jsonpretty=1". I've noticed that for a given flag that putting the flag name, followed by "=1" (i.e. v=1) works.[/QUOTE]

No, sorry, that doesn't work.

bsquared 2022-08-17 12:49

For me, either of the following work in yafu.ini:
[CODE]jsonpretty
jsonpretty=1[/CODE]

Things to check:
1) make sure you have the latest .exe. I updated the executable without tweaking the version number, so you'd have needed to re-download it.
2) If the statement is the last line of the file, make sure it has a newline/carriage return.
3) I have only tested on linux and windows. I have no idea how something different could change the results but stranger things have happened.

bsquared 2022-08-17 13:06

[QUOTE=EdH;611283]Quick question: Can YAFU pass on GPU use for Msieve calls. I'm not seeing it in the docs. I thought I had in the past.[/QUOTE]

It doesn't have the ability to pass on user-input arguments to msieve as strings. If the gpu can be used without any special argument strings, and you build msieve-gpu and link it into yafu like normal, then I suppose it would work. Otherwise you'd have to modify the source.

I have only rarely used msieve-gpu... if you can provide me the special argument strings needed (if any) then I can maybe add an option to use it. Also if using the gpu changes other things like norms or min_e.

EdH 2022-08-17 13:31

[QUOTE=bsquared;611635]It doesn't have the ability to pass on user-input arguments to msieve as strings. If the gpu can be used without any special argument strings, and you build msieve-gpu and link it into yafu like normal, then I suppose it would work. Otherwise you'd have to modify the source.

I have only rarely used msieve-gpu... if you can provide me the special argument strings needed (if any) then I can maybe add an option to use it. Also if using the gpu changes other things like norms or min_e.[/QUOTE]Thanks! I think just adding [C]-g 0[/C] to the msieve call should work, but I'll play and get back with you. I just wanted to make sure it wasn't already there and I was missing it. (Maybe there's a way to have msieve default to GPU if available...)

kotenok2000 2022-08-17 16:35

1 Attachment(s)
Can you tell me why doesn't siqs output show on screen when using this perl script?

[CODE]==== sieving in progress (8 threads): 101408 relations needed ====
==== Press ctrl-c to abort and save state ====
2789 rels found: 2583 full + 206 from 172505 partial, (9836.93 rels/sec)))[/CODE]

When running through ali.pl i only see this:
[CODE]==== sieving in progress (8 threads): 101408 relations needed ====
==== Press ctrl-c to abort and save state ====[/CODE]

EdH 2022-08-17 18:09

It looks like if I compile Msieve with CUDA support, it doesn't make its way into YAFU. I get CUDA errors. I would have to run down everything one-by-one, which I can't currently see myself doing real soon. I might play at it again at some point, but not for now.

Thanks for all.

James Heinrich 2022-08-18 00:54

[QUOTE=James Heinrich;610801]I wrote a PHP script to convert [c]factor.log[/c] to [c]factor.json[/c][/QUOTE]In case it's useful to anyone else, I wrote a small PHP script to convert [c]factor.json[/c] to simple [c]composite=factor1*factor2[/c] style for submission to factordb.com (which doesn't currently support the new YAFU factor.json format).[code]<?php
// parse YAFU factor.json lines into COMPOSITE=FACTOR*FACTOR result lines for factordb.com

if ((count($_SERVER['argv']) != 2) || !preg_match('#\\.json$#i', $_SERVER['argv'][1])) {
echo 'Usage: php '.basename(__FILE__).' factor.json'."\n";
} elseif (!file_exists($_SERVER['argv'][1])) {
echo $_SERVER['argv'][1].' does not exist'."\n";
}

$result_lines = '';
foreach (explode("\n", file_get_contents($_SERVER['argv'][1])) as $linecounter => $line) {
if ($line = trim($line)) {
if ((substr($line, 0, 1) == '{') && (substr($line, -1, 1) == '}')) {
if ($JSON = json_decode($line, true)) {
$result_lines .= $JSON['input-decimal'].'='.implode('*', $JSON['factors-prime']).(!empty($JSON['factors-composite']) ? '*'.implode('*', $JSON['factors-composite']) : '')."\n";
} else {
echo 'error '.__LINE__.': json_decode() failed on line '.($linecounter + 1).' of '.$json_filename.':'."\n".$line."\n\n"; exit(1);
}
} else {
echo 'error '.__LINE__.': unexpected line '.($linecounter + 1).' in '.$json_filename.':'."\n".$line."\n\n"; exit(1);
}
}
}
echo $result_lines."\n";[/code]

chris2be8 2022-08-18 15:42

[QUOTE=kotenok2000;611643]Can you tell me why doesn't siqs output show on screen when using this perl script?
[/QUOTE]

From ali.pl.txt:
[code]
open(YAFU, "./yafu \"factor($composite)\" -threads $yafuthreads|") or die "Couldn't start yafu!";
while (<YAFU>) {
print "$_";
[/code]

AFAIK output from yafu get's buffered somewhere, it will only be passed to your script when yafu has output a complete line. But the progress will be output by yafu as some text, without a newline, then control chars to get back to the start of the *same* line on screen. So your script won't get anything to print until yafu has finished sieving.

I don't know any way round this though.

kotenok2000 2022-08-18 15:52

1 Attachment(s)
But gnfs output is displayed correctly.
Maybe some output is send to stderr and some to stdout and perl relays only stdout?

bsquared 2022-08-18 16:49

gnfs output contains newlines.

siqs progress output does not... only carriage returns (\r)

kotenok2000 2022-08-18 16:51

Can yafu be modified so siqs output also contains newlines?

EdH 2022-08-18 17:50

[QUOTE=kotenok2000;611702]Can yafu be modified so siqs output also contains newlines?[/QUOTE]You can modify the YAFU code, but your output would scroll everything continuously. I don't have time to search right now, but there is a way to empty the buffer periodically with either Python or Perl. I don't remember which I was working with for that at the time. That is where you should look for a solution. If I can, I'll try to find more info later. I'm kind of tied up right now.

Another option you might consider is calling YAFU via a system call rather than a Perl call. That should give you the standard YAFU output. You may need to handle the results a bit differently, though.

ETA: You could look at "Perl STDOUT->autoflush(1)" to see if it might help.

kotenok2000 2022-08-19 10:28

Does yafu 2 still support old cpus without avx?

chris2be8 2022-08-19 15:43

Your best option is probably to call yafu as [c]system("./yafu \"factor($composite)\" -threads $yafuthreads -logfile yafu.log");[/c], then parse yafu.log to find the factors once it has finished. Then delete yafu.log etc to tidy up. That's basically what my scripts to factor small composites in factordb do.

BudgieJane 2022-08-19 22:50

[QUOTE=bsquared;611633]For me, either of the following work in yafu.ini:
[CODE]jsonpretty
jsonpretty=1[/CODE]

Things to check:
1) make sure you have the latest .exe. I updated the executable without tweaking the version number, so you'd have needed to re-download it.
2) If the statement is the last line of the file, make sure it has a newline/carriage return.
3) I have only tested on linux and windows. I have no idea how something different could change the results but stranger things have happened.[/QUOTE]

I have the latest .exe: if I put -jsonpretty on the command line it works.
The statement is not the last line of the .ini file.
I'm using windows.

[Edited at quarter past midnight.]
Because I'm running this by means of a system call from another program, the yafu.ini I need to amend is the one in the other program's home directory and not the one in yafu-2.09. As soon as I add jsonpretty to that file, all works as it should.

How can I tell yafu-2.09 to use its own yafu.ini file instead of the yafu.ini file in the current directory?

NyanCat 2022-08-26 12:44

According to the source code, it always chooses the yafu.ini in the working directory. There are three ways to solve this:
1. cd to the yafu directory before running it
2. Copy the default yafu.ini to your directory
3. Change the source code to always find yafu.ini in your directory
To be exact, you can change the path in top/driver.c Line 110
ini_success = readINI("yafu.ini", options);

kotenok2000 2022-08-27 00:07

Sometimes yafu gets stuck siqs sieving on repeat and I have to kill yafu.exe
It reports

[CODE]08/27/22 01:31:03, trial division touched 0 sieve locations out of 0
08/27/22 01:31:03, total reports = 0, total surviving reports = 0
08/27/22 01:31:03, total blocks sieved = 0, avg surviving reports per block = -nan(ind)
08/27/22 01:31:03, dlp-ecm: 0 failures, 0 attempts, 0 outside range, 0 prp, 0 useful[/CODE]
I use fdb_comp_evaluator.pl from [URL="https://github.com/AntonioPK/Fdb"]https://github.com/AntonioPK/Fdb[/URL]

EdH 2022-08-28 13:40

Another member asked me to test a failure and I can replicate it with both 2.07 and 2.09. The composite is 9649340769776349618630915417390658987772498722136713669954798667326094136661 and it has the short form (8^85-2)/6 per factordb:[code]YAFU Version 2.09
Built with GCC 9
Using GMP-ECM 7.0.5-dev, Powered by GMP 6.2.0
Detected Intel(R) Core(TM) i7-2600 CPU @ 3.40GHz
Detected L1 = 32768 bytes, L2 = 8388608 bytes, CL = 64 bytes
Using 1 random witness for Rabin-Miller PRP checks
Cached 664579 primes; max prime is 9999991
Parsed yafu.ini from /home/math96/Math/yafu

===============================================================
======= Welcome to YAFU (Yet Another Factoring Utility) =======
======= bbuhrow@gmail.com =======
======= Type help at any time, or quit to quit =======
===============================================================

>> factor(9649340769776349618630915417390658987772498722136713669954798667326094136661)
fac: factoring 9649340769776349618630915417390658987772498722136713669954798667326094136661
fac: using pretesting plan: normal
fac: using specified qs/gnfs crossover of 100 digits
fac: using specified qs/snfs crossover of 75 digits
div: primes less than 10000
fmt: 1000000 iterations
rho: x^2 + 3, starting 1000 iterations on C76
rho: x^2 + 2, starting 1000 iterations on C76
rho: x^2 + 1, starting 1000 iterations on C76
nfs: searching for brent special forms...
nfs: input divides 2^254 - 1
nfs: snfs form detection took 0.000103 seconds
nfs: snfs form detection took 0.000061 seconds
gen: found primitive cofactor < input number:
gen: 170141183460469231731687303715884105727
fac: ecm effort reduced from 23.38 to 18.19: input has snfs form
pm1: starting B1 = 150K, B2 = gmp-ecm default on C76
nfs: snfs form detection took 0.000109 seconds
gen: found primitive cofactor < input number:
gen: 170141183460469231731687303715884105727
fac: ecm effort reduced from 23.38 to 18.19: input has snfs form
ecm: 30/30 curves on C76, B1=2k, B2=gmp-ecm default
nfs: snfs form detection took 0.000069 seconds
gen: found primitive cofactor < input number:
gen: 170141183460469231731687303715884105727
fac: ecm effort reduced from 23.38 to 18.19: input has snfs form
ecm: 45/45 curves on C76, B1=11k, B2=gmp-ecm default
nfs: snfs form detection took 0.000073 seconds
gen: found primitive cofactor < input number:
gen: 170141183460469231731687303715884105727
fac: ecm effort reduced from 23.38 to 18.19: input has snfs form
nfs: commencing nfs on c76: 9649340769776349618630915417390658987772498722136713669954798667326094136661
nfs: searching for brent special forms...
nfs: input divides 2^254 - 1
nfs: snfs form detection took 0.000091 seconds
gen: found primitive cofactor < input number:
gen: 170141183460469231731687303715884105727
nfs: found 4 polynomials, selecting best
nfs: using provided snfs difficulty 38.230809
nfs: using provided snfs difficulty 38.531839
nfs: using provided snfs difficulty 38.230809
nfs: commencing algebraic side lattice sieving over range: 155000 - 160000
nfs: commencing algebraic side lattice sieving over range: 180000 - 185000
nfs: commencing algebraic side lattice sieving over range: 160000 - 165000
nfs: commencing algebraic side lattice sieving over range: 170000 - 175000
nfs: commencing algebraic side lattice sieving over range: 150000 - 155000
nfs: commencing algebraic side lattice sieving over range: 175000 - 180000
nfs: commencing algebraic side lattice sieving over range: 165000 - 170000
nfs: commencing algebraic side lattice sieving over range: 185000 - 190000
*** Error in `../ggnfs/bin/gnfs-lasieve4I11e': realloc(): invalid next size: 0x00000000010b52f0 ***
======= Backtrace: =========
[0x47fc21]
[0x48b166]
[0x48c489]
[0x40bb25]
[0x40acf9]
[0x406167]
[0x467ba6]
[0x467d9a]
[0x4083f9]
======= Memory map: ========
00400000-0055e000 r-xp 00000000 08:01 6693409 /home/math96/Math/ggnfs/bin/gnfs-lasieve4I11e
0075d000-00760000 rwxp 0015d000 08:01 6693409 /home/math96/Math/ggnfs/bin/gnfs-lasieve4I11e
00760000-007f5000 rwxp 00000000 00:00 0
00d3d000-01355000 rwxp 00000000 00:00 0 [heap]
7f37dc000000-7f37dc024000 rwxp 00000000 00:00 0
7f37dc024000-7f37e0000000 ---p 00000000 00:00 0
7f37e1f5f000-7f37e45a2000 rwxp 00000000 00:00 0
7fff9ca0d000-7fff9ca2e000 rwxp 00000000 00:00 0 [stack]
7fff9cad1000-7fff9cad4000 r--p 00000000 00:00 0 [vvar]
7fff9cad4000-7fff9cad5000 r-xp 00000000 00:00 0 [vdso]
ffffffffff600000-ffffffffff601000 --xp 00000000 00:00 0 [vsyscall]
*** Error in `../ggnfs/bin/gnfs-lasieve4I11e': realloc(): invalid next size: 0x0000000002672610 ***
======= Backtrace: =========
[0x47fc21]
[0x48b166]
[0x48c489]
[0x40bb25]
[0x40acf9]
[0x406167]
[0x467ba6]
[0x467d9a]
[0x4083f9]
======= Memory map: ========
00400000-0055e000 r-xp 00000000 08:01 6693409 /home/math96/Math/ggnfs/bin/gnfs-lasieve4I11e
0075d000-00760000 rwxp 0015d000 08:01 6693409 /home/math96/Math/ggnfs/bin/gnfs-lasieve4I11e
00760000-007f5000 rwxp 00000000 00:00 0
022f8000-02913000 rwxp 00000000 00:00 0 [heap]
7f8240000000-7f8240024000 rwxp 00000000 00:00 0
7f8240024000-7f8244000000 ---p 00000000 00:00 0
7f8245874000-7f8247ec1000 rwxp 00000000 00:00 0
7ffcaa002000-7ffcaa023000 rwxp 00000000 00:00 0 [stack]
7ffcaa1b7000-7ffcaa1ba000 r--p 00000000 00:00 0 [vvar]
7ffcaa1ba000-7ffcaa1bb000 r-xp 00000000 00:00 0 [vdso]
ffffffffff600000-ffffffffff601000 --xp 00000000 00:00 0 [vsyscall]
Aborted (core dumped)
Aborted (core dumped)
*** Error in `../ggnfs/bin/gnfs-lasieve4I11e': realloc(): invalid next size: 0x0000000001c42f20 ***
======= Backtrace: =========
[0x47fc21]
[0x48b166]
[0x48c489]
[0x40bb25]
[0x40acf9]
[0x406167]
[0x467ba6]
[0x467d9a]
[0x4083f9]
======= Memory map: ========
00400000-0055e000 r-xp 00000000 08:01 6693409 /home/math96/Math/ggnfs/bin/gnfs-lasieve4I11e
0075d000-00760000 rwxp 0015d000 08:01 6693409 /home/math96/Math/ggnfs/bin/gnfs-lasieve4I11e
00760000-007f5000 rwxp 00000000 00:00 0
01853000-01f59000 rwxp 00000000 00:00 0 [heap]
7f4d38000000-7f4d38024000 rwxp 00000000 00:00 0
7f4d38024000-7f4d3c000000 ---p 00000000 00:00 0
7f4d3c1b3000-7f4d3e7ec000 rwxp 00000000 00:00 0
7ffc33d13000-7ffc33d34000 rwxp 00000000 00:00 0 [stack]
7ffc33d82000-7ffc33d85000 r--p 00000000 00:00 0 [vvar]
7ffc33d85000-7ffc33d86000 r-xp 00000000 00:00 0 [vdso]
ffffffffff600000-ffffffffff601000 --xp 00000000 00:00 0 [vsyscall]
*** Error in `../ggnfs/bin/gnfs-lasieve4I11e': realloc(): invalid next size: 0x0000000001960e50 ***
======= Backtrace: =========
[0x47fc21]
[0x48b166]
[0x48c489]
[0x40bb25]
[0x40acf9]
[0x406167]
[0x467ba6]
[0x467d9a]
[0x4083f9]
======= Memory map: ========
00400000-0055e000 r-xp 00000000 08:01 6693409 /home/math96/Math/ggnfs/bin/gnfs-lasieve4I11e
0075d000-00760000 rwxp 0015d000 08:01 6693409 /home/math96/Math/ggnfs/bin/gnfs-lasieve4I11e
00760000-007f5000 rwxp 00000000 00:00 0
01431000-01b71000 rwxp 00000000 00:00 0 [heap]
7ff054000000-7ff054024000 rwxp 00000000 00:00 0
7ff054024000-7ff058000000 ---p 00000000 00:00 0
7ff05ae9a000-7ff05d4b4000 rwxp 00000000 00:00 0
7ffc503c6000-7ffc503e7000 rwxp 00000000 00:00 0 [stack]
7ffc503ef000-7ffc503f2000 r--p 00000000 00:00 0 [vvar]
7ffc503f2000-7ffc503f3000 r-xp 00000000 00:00 0 [vdso]
ffffffffff600000-ffffffffff601000 --xp 00000000 00:00 0 [vsyscall]
Aborted (core dumped)
Aborted (core dumped)
*** Error in `../ggnfs/bin/gnfs-lasieve4I11e': realloc(): invalid next size: 0x00000000021c9770 ***
======= Backtrace: =========
[0x47fc21]
[0x48b166]
[0x48c489]
[0x40bb25]
[0x40acf9]
[0x406167]
[0x467ba6]
[0x467d9a]
[0x4083f9]
======= Memory map: ========
00400000-0055e000 r-xp 00000000 08:01 6693409 /home/math96/Math/ggnfs/bin/gnfs-lasieve4I11e
0075d000-00760000 rwxp 0015d000 08:01 6693409 /home/math96/Math/ggnfs/bin/gnfs-lasieve4I11e
00760000-007f5000 rwxp 00000000 00:00 0
01b3b000-02488000 rwxp 00000000 00:00 0 [heap]
7f2928000000-7f2928024000 rwxp 00000000 00:00 0
7f2928024000-7f292c000000 ---p 00000000 00:00 0
7f292e8cb000-7f2930f21000 rwxp 00000000 00:00 0
7fff6ad49000-7fff6ad6a000 rwxp 00000000 00:00 0 [stack]
7fff6ad8e000-7fff6ad91000 r--p 00000000 00:00 0 [vvar]
7fff6ad91000-7fff6ad92000 r-xp 00000000 00:00 0 [vdso]
ffffffffff600000-ffffffffff601000 --xp 00000000 00:00 0 [vsyscall]
Aborted (core dumped)
total yield: 169943, q=160031 (0.00002 sec/rel)
total yield: 179959, q=150193 (0.00002 sec/rel)
total yield: 171864, q=165049 (0.00002 sec/rel)


***factors found***


***co-factor***
C76 = 9649340769776349618630915417390658987772498722136713669954798667326094136661[/code]Due to residual files, subsequent runs provide one factor, but not both:[code]
YAFU Version 2.09
Built with GCC 9
Using GMP-ECM 7.0.5-dev, Powered by GMP 6.2.0
Detected Intel(R) Core(TM) i7-2600 CPU @ 3.40GHz
Detected L1 = 32768 bytes, L2 = 8388608 bytes, CL = 64 bytes
Using 1 random witness for Rabin-Miller PRP checks
Cached 664579 primes; max prime is 9999991
Parsed yafu.ini from /home/math96/Math/yafu

===============================================================
======= Welcome to YAFU (Yet Another Factoring Utility) =======
======= bbuhrow@gmail.com =======
======= Type help at any time, or quit to quit =======
===============================================================

>> factor(9649340769776349618630915417390658987772498722136713669954798667326094136661)
fac: factoring 9649340769776349618630915417390658987772498722136713669954798667326094136661
fac: using pretesting plan: normal
fac: using specified qs/gnfs crossover of 100 digits
fac: using specified qs/snfs crossover of 75 digits
nfs: searching for brent special forms...
nfs: input divides 2^254 - 1
nfs: snfs form detection took 0.000082 seconds

starting SIQS on c39: 170141183460469231731687303715884105727
Total factoring time = 0.1513 seconds


***factors found***

P39 = 170141183460469231731687303715884105727

ans = 1[/code]I tested the siever that was reported faulty (with a separate test poly) and it ran fine.

Thanks for all your work.

James Heinrich 2022-08-28 14:08

[QUOTE=EdH;612233]Another member asked me to test a failure and I can replicate it with both 2.07 and 2.09[/QUOTE]For what it's worth, it works fine on v1.34.5:[code]Starting factorization of 9649340769776349618630915417390658987772498722136713669954798667326094136661
starting SIQS on c76: 9649340769776349618630915417390658987772498722136713669954798667326094136661
commencing Lanczos iteration
memory use: 2.5 MB
lanczos halted after 344 iterations (dim = 21633)
recovered 16 nontrivial dependencies
prp39 = 170141183460469231731687303715884105727
prp38 = 56713727820156410577229101238628035243
Lanczos elapsed time = 1.2290 seconds.
Sqrt elapsed time = 0.0180 seconds.
SIQS elapsed time = 10.4296 seconds.
Total factoring time = 22.3813 seconds[/code]

kotenok2000 2022-08-28 14:22

Maybe you compiled gnfs incorrectly?
What if you copy gnfs-lasieve files from yafu boinc project slot directory?
They have linux compiled executables. [url]https://yafu.myfirewall.org/yafu/apps.php[/url]

EdH 2022-08-28 14:48

[QUOTE=James Heinrich;612236]For what it's worth, it works fine on v1.34.5:[/QUOTE]Thanks! I hadn't tried version 1.
[QUOTE=kotenok2000;612237]Maybe you compiled gnfs incorrectly?. . .[/QUOTE]Thanks, but it also fails with the siever that bsquared compiled.

chris2be8 2022-08-28 15:32

(8^85-2)/6 simplifies to (2^254-1)/3 which factors to (2^127-1)*(2^127+1)/3.

The problem was probably caused by the fact the expression had algebraic factors.

bsquared 2022-08-28 15:53

If you look closely at EdH's post of the output you'll see:

[CODE]nfs: commencing nfs on c76: 9649340769776349618630915417390658987772498722136713669954798667326094136661
nfs: searching for brent special forms...
nfs: input divides 2^254 - 1
nfs: snfs form detection took 0.000091 seconds
gen: found primitive cofactor < input number:
gen: 170141183460469231731687303715884105727[/CODE]

Which shows that it recognizes the form 2^254-1 and that it found the algebraic factor (2^127-1). What I guess I hadn't anticipated is that this would complete the factorization. I can patch that up. The ggnfs sievers must have issues when asked to factor a small prime number, or the parameters are crazy because the input (a p38) is so small.

1.34.5 works because the siqs/snfs crossovers are apparently set differently so it proceeds immediately to siqs without doing the algebraic factor detection.

LaurV 2022-09-05 09:41

In yafu.ini from git, there is no B1pp1 nor B1pp2, but the B1pm1 and B1pm2 appear two times.

Intentional, or copy/paste error?

storm5510 2022-09-06 18:41

For some reason, the latest release of YAFU will run on my i7 system, but not a Xeon I have. Both Windows 10 and 16 GB of RAM.

It displays its title screen and some other info then drops back to the command prompt. No error messages. :confused:

James Heinrich 2022-09-06 18:48

[QUOTE=storm5510;612806]i7 system, but not a Xeon I have[/QUOTE]"i7" and "Xeon" are both nebulous descriptors that could describe any number of CPUs over decades. Perhaps specific model numbers would provide more of a clue as to perhaps which features are not supported on your "Xeon".

storm5510 2022-09-06 23:17

[QUOTE=James Heinrich;612807]"i7" and "Xeon" are both nebulous descriptors that could describe any number of CPUs over decades. Perhaps specific model numbers would provide more of a clue as to perhaps which features are not supported on your "Xeon".[/QUOTE]

E5-1620 Sandy Bridge EP.

Version 1.34.5 runs without issues. Nothing later will.

kotenok2000 2022-09-07 09:00

[url]https://www.cpu-world.com/CPUs/Xeon/Intel-Xeon%20E5-1620.html[/url]
Maybe it requires avx2?

bsquared 2022-09-07 17:47

It should work on systems without AVX2, but I don't have any of those to test on anymore, so apparently it doesn't. Based on where it crashes it would seem the problem lies in the ysieve library, since one of the first things yafu does is create and store a small list of primes.

kruoli 2022-09-07 18:12

Yes, I have analyzed this before: :smile:

[QUOTE=kruoli;589431]When running the current build on GIT on an Ivy Bridge processor, I get an invalid instruction error (MULX, since Ivy Bridge does not support BMI(2)) at 0x000000013F571B43. This happens when the prompt ([C]>> [/C]) is already displayed. Is the processor too old to be supported out of the box?[/QUOTE]

James Heinrich 2022-09-07 20:52

1 Attachment(s)
I know my [url=https://www.cpu-world.com/CPUs/Core_i7/Intel-Core%20i7-3930K.html]i7-3930K[/url] (also Sandy Bridge) doesn't run YAFU 2.x, but I had always assumed that was Windows-7 thing. I guess it's (also?) a hardware thing.

(is it sad that my main daily system is nearing 11 years old? :paul:)

kruoli 2022-09-07 21:38

[QUOTE=James Heinrich;612905](is it sad that my main daily system is nearing 11 years old? :paul:)[/QUOTE]

No, I won't give up on my one voluntarily!

storm5510 2022-09-08 15:54

[QUOTE=James Heinrich;612905]I know my [url=https://www.cpu-world.com/CPUs/Core_i7/Intel-Core%20i7-3930K.html]i7-3930K[/url] (also Sandy Bridge) doesn't run YAFU 2.x, but I had always assumed that was Windows-7 thing. I guess it's (also?) a hardware thing.

(is it sad that my main daily system is nearing 11 years old? :paul:)[/QUOTE]

Only my i7-7700 is AVX2. The Xeon and i5 are not. These two date back to 2012. The i7 is four years old, (2018). I sourced the parts and built it myself. The Xeon can run Windows 10 or Ubuntu 20.04. I doubt the i5 could run Windows 10. It runs Windows 7 well enough.

kotenok2000 2022-09-19 19:20

My amd fx 6100 ran windows 10

wombatman 2022-09-20 04:27

I'm running into the following error while trying to compile the latest YAFU:

[CODE]/usr/bin/ld: ./libysiqs.a(SIQS.o): in function `siqs_static_init':
/home/wombat/yafu/factor/qs/SIQS.c:2309: undefined reference to `nextRoots_32k_avx2_intrin'[/CODE]

Compilation instruction is "make USE_AVX2 NFS=1". Everything else seems to be building properly as the above is the only error.

bsquared 2022-09-20 13:10

[QUOTE=wombatman;613736]I'm running into the following error while trying to compile the latest YAFU:

[CODE]/usr/bin/ld: ./libysiqs.a(SIQS.o): in function `siqs_static_init':
/home/wombat/yafu/factor/qs/SIQS.c:2309: undefined reference to `nextRoots_32k_avx2_intrin'[/CODE]

Compilation instruction is "make USE_AVX2 NFS=1". Everything else seems to be building properly as the above is the only error.[/QUOTE]

You'll need to add USE_BMI2=1 to the make line as well. Most cpu's that have avx2 also have bmi2... if yours doesn't then you'll have to remove both the avx2 and bmi2 options.

bsquared 2022-09-20 13:49

v. 2.10 available
 
I checked in version 2.10.

Hopefully fixes the recently reported crashing issue right after yafu starts on systems without BMI1/2 (this is still not something I can directly test, so let me know if the problem still exists).

Also fixes the problem where primitive/algebraic factor detection left a prime number that NFS tried to factor. Now if that happens the factorization should finish.

The new/improved microecm is also included (added to the makefile/build). Thanks again to Jeff Hurchalla.

Almost forgot... external GPU-ECM binaries are now supported, optionally with Seth Trosi's CGBN enhancements. new options: "gpucurves", "cgbn", "use_gpuecm", "use_gpudev". Note that if you are using GPU-ECM with "factor()", then you'll probably want to make use of the "ext_ecm" option. This option sets when to switch over from internal non-gpu ECM to the external GPU-ECM binary. Otherwise, for example, factor() will try to use potentially thousands of GPU curves at B1=50000 when it only needs a few hundred. If you have AVX-ECM, then that will still be used as the internal ECM version before switching to external GPU-ECM.

Finally, I fixed several memory leaks after updating to a new version of valgrind. Either valgrind doesn't know how to deal with avx512 instructions or I don't know how to use it correctly, so the memory leaks were investigated without any of that code path... possibly some still exist there. But overall the situation should be improved.

If you are building from source you will also need to get the newest version of ytools/ysieve. Some of the memory leaks and fixes were in the ysieve library.

kruoli 2022-09-20 15:26

Does it require a self-compiled binary for the fix for Ivy Bridge etc.? The .exe in your GIT repository still crashed on startup on Ivy Bridge, Windows 7.

James Heinrich 2022-09-20 15:29

[QUOTE=kruoli;613764]Does it require a self-compiled binary for the fix for Ivy Bridge etc.?
The .exe in your GIT repository still crashed on startup on Ivy Bridge, Windows 7.[/QUOTE]It was my understanding that [i]none[/i] of the v2.0 YAFU will ever work on Windows 7. Crashing on old CPUs under Win10 I think was a different issue.

For what it's worth v2.10 seems to run fine on my i3-8100/Win10.

bsquared 2022-09-20 15:35

[QUOTE=James Heinrich;613765]It was my understanding that [i]none[/i] of the v2.0 YAFU will ever work on Windows 7. Crashing on old CPUs under Win10 I think was a different issue.

For what it's worth v2.10 seems to run fine on my i3-8100/Win10.[/QUOTE]

I don't know why it would crash on windows 7 and not windows 10. But my understanding of the dependencies of Visual Studio 2022 is very murky. Maybe it requires win10....

My hope is that it is a cpu-feature problem, not a windows version problem. But I can't directly test either one.

I highly recommend WSL/ubuntu if you're running windows. IIRC that is available for windows 7?

It looks like Ivy Bridge doesn't have either AVX2 or BMI2. But this *should* be detected and those features not used by either ysieve or yafu. Can you start up with -v -v -v -v and see if anything extra is printed?

James Heinrich 2022-09-20 15:54

2 Attachment(s)
[QUOTE=bsquared;613769]I don't know why it would crash on windows 7 and not windows 10.
My hope is that it is a cpu-feature problem, not a windows version problem. But I can't directly test either one.[/QUOTE]I'm happy to do any testing you want on my i7-3930K (Sandy Bridge).

[QUOTE=bsquared;613769]I highly recommend WSL/ubuntu if you're running windows. IIRC that is available for windows 7?[/quote]No. If it's the WSL I think you mean, [url=https://en.wikipedia.org/wiki/Windows_Subsystem_for_Linux]WSL is[/url] [c]a compatibility layer for running Linux binary executables (in ELF format) natively on Windows 10, Windows 11, and Windows Server 2019[/c].

[QUOTE=bsquared;613769]It looks like Ivy Bridge doesn't have either AVX2 or BMI2. But this *should* be detected and those features not used by either ysieve or yafu. Can you start up with -v -v -v -v and see if anything extra is printed?[/QUOTE]Quadverbosity makes no difference that I notice. See attached. Crash details:[code]Problem signature:
Problem Event Name: APPCRASH
Application Name: yafu-x64.exe
Application Version: 0.0.0.0
Application Timestamp: 6329c003
Fault Module Name: yafu-x64.exe
Fault Module Version: 0.0.0.0
Fault Module Timestamp: 6329c003
Exception Code: c000001d
Exception Offset: 0000000000001300
OS Version: 6.1.7601.2.1.0.256.48
Locale ID: 1033
Additional Information 1: e63c
Additional Information 2: e63cb477893153aa77789ca6de3c6a1b
Additional Information 3: 2fdb
Additional Information 4: 2fdb2bf6e03bebbe6df1b584e671261b[/code]

kruoli 2022-09-20 15:57

As far as I know, WSL is not availible for Windows 7.

Requested test:
[CODE]C:\tmp>yafu -v -v -v -v
warning: could not open yafu.ini, no options parsed


YAFU Version 2.10
Built with Microsoft Visual Studio 1931
Using GMP-ECM 7.0.4, Powered by MPIR 3.0.0
Detected Intel(R) Core(TM) i7-3630QM CPU @ 2.40GHz
Detected L1 = 32768 bytes, L2 = 6291456 bytes, CL = 64 bytes
Using 1 random witness for Rabin-Miller PRP checks
Cached 664579 primes; max prime is 9999991
Could not parse yafu.ini from C:\tmp

===============================================================
======= Welcome to YAFU (Yet Another Factoring Utility) =======
======= bbuhrow@gmail.com =======
======= Type help at any time, or quit to quit =======
===============================================================

>>[/CODE]
...and then crashed as usual.
[C]Error at 0x000000013F641300 in yafu.exe: 0xC000001D: Illegal Instruction[/C]

bsquared 2022-09-20 16:17

Can you share the last 10-15 lines of session.log after an attempted startup?

storm5510 2022-09-20 17:19

[QUOTE=bsquared;612893]It should work on systems without AVX2, but I don't have any of those to test on anymore, so apparently it doesn't. Based on where it crashes it would seem the problem lies in the ysieve library, since one of the first things yafu does is create and store a small list of primes.[/QUOTE]

Link please. I can't test it if I can't find it...

bsquared 2022-09-20 17:25

[QUOTE=storm5510;613777]Link please. I can't test it if I can't find it...[/QUOTE]

[URL="https://github.com/bbuhrow/yafu/blob/master/yafu-x64.exe"]Here[/URL].

By the way, I now suspect the linked GMP library as the problem, since the initial batch of primes is found just fine. The illegal instruction could lie in GMP's random number generator setup.

James Heinrich 2022-09-20 18:03

[QUOTE=bsquared;613772]Can you share the last 10-15 lines of session.log after an attempted startup?[/QUOTE]Empty. If I delete [c]session.log[/c] it gets created as a zero-byte file but nothing is written to it before the crash (or after for that matter).

storm5510 2022-09-20 23:30

[QUOTE=bsquared;613778][URL="https://github.com/bbuhrow/yafu/blob/master/yafu-x64.exe"]Here[/URL].
[/QUOTE]

Thank you for the reply. The link appears to be to a Linux build. I read some install instructions [B]EdH[/B] wrote and it sailed over my head. I have been running a Windows build, v2.09. It does fine. My Ubuntu box is usually busy with [I]CADO-NFS[/I].

James Heinrich 2022-09-21 01:19

[QUOTE=storm5510;613796]The link appears to be to a Linux build[/QUOTE]No, it's the Windows build. Note however that it's a link to a webpage from which you can download it, do not save-link-as directly -- open the link in your browser then click the Download button.
Or download it from [url]https://download.mersenne.ca/YAFU[/url] if you prefer.

storm5510 2022-09-21 14:16

[QUOTE=James Heinrich;613799]No, it's the Windows build. Note however that it's a link to a webpage from which you can download it, do not save-link-as directly -- open the link in your browser then click the Download button.
Or download it from [url]https://download.mersenne.ca/YAFU[/url] if you prefer.[/QUOTE]

Thank you. Most kind. :smile:

bsquared 2022-09-21 18:14

[QUOTE=bsquared;613778][URL="https://github.com/bbuhrow/yafu/blob/master/yafu-x64.exe"]Here[/URL].

By the way, I now suspect the linked GMP library as the problem, since the initial batch of primes is found just fine. The illegal instruction could lie in GMP's random number generator setup.[/QUOTE]

I checked in yafu-x64-gc.exe, which I created by linking against a generic C version of MPIR-3.0.0. Please test it and see if it's any better.

James Heinrich 2022-09-21 19:35

1 Attachment(s)
[QUOTE=bsquared;613841]I checked in yafu-x64-gc.exe, which I created by linking against a generic C version of MPIR-3.0.0. Please test it and see if it's any better.[/QUOTE]It's no better for me. Same crash (after startup) and session.log is still empty.

WraithX 2022-09-21 20:48

Btw, just like [URL="https://mersenneforum.org/showpost.php?p=586255&postcount=175"]last time[/URL] (post 175 in this thread) I am able to run yafu 2.10 on my Windows 7 x64 computer without any problems.

[CODE]
D:\yafu\yafu_2.10>yafu-x64.exe
warning: could not open yafu.ini, no options parsed


YAFU Version 2.10
Built with Microsoft Visual Studio 1931
Using GMP-ECM 7.0.4, Powered by MPIR 3.0.0
Detected Intel(R) Xeon(R) CPU E5-2687W v4 @ 3.00GHz
Detected L1 = 32768 bytes, L2 = 31457280 bytes, CL = 64 bytes
Using 1 random witness for Rabin-Miller PRP checks
Cached 664579 primes; max prime is 9999991
Could not parse yafu.ini from D:\yafu\yafu_2.10

===============================================================
======= Welcome to YAFU (Yet Another Factoring Utility) =======
======= bbuhrow@gmail.com =======
======= Type help at any time, or quit to quit =======
===============================================================

>> aprcl(2^31-1)

P = 5, Q = 11 (88.89%)
Input is prime. P10

ans = 2147483647

>> factor(10^19+49)

fac: factoring 10000000000000000049
fac: using pretesting plan: normal
fac: no tune info: using qs/gnfs crossover of 95 digits
fac: no tune info: using qs/snfs crossover of 75 digits
div: primes less than 10000
fmt: 1000000 iterations
rho: x^2 + 3, starting 1000 iterations on C20
Total factoring time = 0.0312 seconds


***factors found***

P10 = 8532383917
P10 = 1172005397

ans = 1
[/CODE]

James Heinrich 2022-09-21 21:11

[QUOTE=WraithX;613853]Btw, just like [URL="https://mersenneforum.org/showpost.php?p=586255&postcount=175"]last time[/URL] (post 175 in this thread) I am able to run yafu 2.10 on my Windows 7 x64 computer without any problems.[/QUOTE]Interesting.

Per your other post I installed [c]Microsoft Visual C++ 2015-2022 Redistributable (x64) 14.32.31332[/c] both x64 and x86 versions but made no difference, same crash.
I don't see much hardware feature difference between your [url=https://www.techpowerup.com/cpu-specs/xeon-e5-2687w.c981]Xeon E5-2687W[/url] and my [url=https://www.techpowerup.com/cpu-specs/core-i7-3930k.c858]i7-3930K[/url].

WraithX 2022-09-21 21:24

I'm not sure if the windbg (windows debugger) can help track down what the illegal instruction is, but it may be worth a shot. You can install it by downloading the [C]Windows SDK for Win 10[/C]

[URL="https://learn.microsoft.com/en-us/windows-hardware/drivers/debugger/debugger-download-tools"]Windows SDK - With WinDbg[/URL]

Download the Windows SDK (it will say for Windows 10, but the installation page says it supports Windows 7 SP1). Then, to just get the debugger, the instructions say:
[CODE]
In the SDK installation wizard, select Debugging Tools for Windows, and deselect all other components.
[/CODE]

At this point it would take more googling to find out how to use the debugger to start the program, run it until the problem occurs, and then find out what instruction is causing the crash. I hope this can help you track down this problem!

James Heinrich 2022-09-21 21:48

[QUOTE=WraithX;613857]Download the Windows SDK (it will say for Windows 10, but the installation page says it supports Windows 7 SP1)[/QUOTE]I got it installed, but can't seem to start it, [c]api-ms-win-core-com-l1-1-0.dll[/c] is missing (not sure if that might be a Win10 thing?).

bsquared 2022-09-21 22:29

I re-uploaded yafu-x64-gc.exe.

This time I did a complete rebuild of the entire dependency chain starting with mpir-3.0.0 (core2), then ecm-7.0.4, ytools, ysieve, and yafu. In all MSVC projects I set the "enable enhanced instruction set" option in the code generation settings tab to "No Enhanced Instructions (/arch/IA32)". Although I don't know if this does anything because I see the message:
"1>cl : command line warning D9002: ignoring unknown option '/arch:IA32'"
scroll by when compiling. weird.

Thanks for testing!

James Heinrich 2022-09-21 23:00

1 Attachment(s)
[QUOTE=bsquared;613862]I re-uploaded yafu-x64-gc.exe[/QUOTE]:w00t:[code]C:\Users\User\Desktop\yafu>yafu-2.10-202209211825-win64-gc.exe

YAFU Version 2.10
Built with Microsoft Visual Studio 1931
Using GMP-ECM 7.0.4, Powered by MPIR 3.0.0
Detected Intel(R) Core(TM) i7-3930K CPU @ 3.20GHz
Detected L1 = 32768 bytes, L2 = 12582912 bytes, CL = 64 bytes
Using 1 random witness for Rabin-Miller PRP checks
Cached 664579 primes; max prime is 9999991
Parsed yafu.ini from C:\Users\User\Desktop\yafu

===============================================================
======= Welcome to YAFU (Yet Another Factoring Utility) =======
======= bbuhrow@gmail.com =======
======= Type help at any time, or quit to quit =======
===============================================================

>> factor(1460028439856908668755525958205639140519011347819950733167960266924037979)
...
Lanczos elapsed time = 1.0690 seconds.
Sqrt elapsed time = 0.0060 seconds.
SIQS elapsed time = 107.7832 seconds.
pretesting / qs ratio was 0.08
Total factoring time = 116.5047 seconds
***factors found***
P36 = 348746364767500102806150303982379243
P37 = 4186505114770932960236858078590076753
ans = 1[/code]However, while running [c]tune()[/c] it crashed (attached) on the second test (according to factor.log):[code]09/21/22 18:56:12, starting SIQS on c60: 349594255864176572614071853194924838158088864370890996447417
09/21/22 18:56:12, random seed: 8752402939882152805
09/21/22 18:56:12, ==== sieve params ====
09/21/22 18:56:12, n = 62 digits, 205 bits
09/21/22 18:56:12, factor base: 3504 primes (max prime = 71333)
09/21/22 18:56:12, single large prime cutoff: 4993310 (70 * pmax)
09/21/22 18:56:12, allocating 2 large prime slices of factor base
09/21/22 18:56:12, buckets hold 2048 elements
09/21/22 18:56:12, large prime hashtables have 65536 bytes
09/21/22 18:56:12, using SSE41 enabled 32k sieve core
09/21/22 18:56:12, sieve interval: 2 blocks of size 32768
09/21/22 18:56:12, polynomial A has ~ 7 factors
09/21/22 18:56:12, using multiplier of 145
09/21/22 18:56:12, using multiplier of 145
09/21/22 18:56:12, using Q2(x) polynomials for kN mod 8 = 1
09/21/22 18:56:12, using SPV correction of 15 bits, starting at offset 34
09/21/22 18:56:12, trial factoring cutoff at 68 bits
09/21/22 18:56:12, ==== sieving started (1 thread) ====
09/21/22 18:56:14, trial division touched 105151 sieve locations out of 1519386624
09/21/22 18:56:14, total reports = 105151, total surviving reports = 61230
09/21/22 18:56:14, total blocks sieved = 46368, avg surviving reports per block = 1.32
09/21/22 18:56:14, 1772 relations found: 1050 full + 722 from 9948 partial, using 11592 polys (183 A polys)
09/21/22 18:56:14, on average, sieving found 0.95 rels/poly and 5668.75 rels/sec
09/21/22 18:56:14, trial division touched 105151 sieve locations out of 1519386624
09/21/22 18:56:14, ==== post processing stage (msieve-1.38) ====
09/21/22 18:56:14, starting SIQS on c65: 34053408309992030649212497354061832056920539397279047809781589871
09/21/22 18:56:14, random seed: 17728683799165162768
09/21/22 18:56:14, ==== sieve params ====
09/21/22 18:56:14, n = 67 digits, 220 bits
09/21/22 18:56:14, factor base: 6384 primes (max prime = 136207)
09/21/22 18:56:14, single large prime cutoff: 10215525 (75 * pmax)
09/21/22 18:56:14, allocating 3 large prime slices of factor base
09/21/22 18:56:14, buckets hold 2048 elements
09/21/22 18:56:14, large prime hashtables have 98304 bytes
09/21/22 18:56:14, using SSE41 enabled 32k sieve core
09/21/22 18:56:14, sieve interval: 2 blocks of size 32768
09/21/22 18:56:14, polynomial A has ~ 8 factors
09/21/22 18:56:14, using multiplier of 47
09/21/22 18:56:14, using multiplier of 47
09/21/22 18:56:14, using Q2(x) polynomials for kN mod 8 = 1
09/21/22 18:56:14, using SPV correction of 20 bits, starting at offset 40
09/21/22 18:56:14, trial factoring cutoff at 74 bits
09/21/22 18:56:14, ==== sieving started (1 thread) ====
09/21/22 18:56:24, trial division touched 383544 sieve locations out of 5371330560
09/21/22 18:56:24, total reports = 383544, total surviving reports = 127599
09/21/22 18:56:24, total blocks sieved = 163920, avg surviving reports per block = 0.78
09/21/22 18:56:24, 1274 relations found: 756 full + 518 from 10324 partial, using 40980 polys (619 A polys)
09/21/22 18:56:24, on average, sieving found 0.27 rels/poly and 1104.51 rels/sec
09/21/22 18:56:24, trial division touched 383544 sieve locations out of 5371330560
09/21/22 18:56:24, ==== post processing stage (msieve-1.38) ====[/code]

bsquared 2022-09-22 00:18

Sweet! Good to see progress.

Actually I realized now that I didn't update the msieve dependency, so maybe that will take care of the error you're seeing now. Might not be able to get that updated until tomorrow.

wombatman 2022-09-22 04:36

USE_BMI2=1 did solve the compilation issue, so thank you for that!

I am also encountering the issue when running tune() with the second siqs number:

[CODE]starting SIQS on c65: 34053408309992030649212497354061832056920539397279047809781589871
overriding small TF cutoff of 20 to 20

==== sieve params ====
n = 67 digits, 220 bits
factor base: 6384 primes (max prime = 136207)
single large prime cutoff: 10215525 (75 * pmax)
allocating 3 large prime slices of factor base
buckets hold 2048 elements
large prime hashtables have 196608 bytes
using AVX2 enabled 32k sieve core
sieve interval: 4 blocks of size 32768
polynomial A has ~ 8 factors
using multiplier of 47
using Q2(x) polynomials for kN mod 8 = 1
using SPV correction of 20 bits, starting at offset 40
trial factoring cutoff at 75 bits

==== sieving in progress (1 thread): 6448 relations needed ====
==== Press ctrl-c to abort and save state ====
750 rels found: 646 full + 104 from 5321 partial, (6828.42 rels/sec)
Max specified relations found

sieve time = 0.0000, relation time = 0.0000, poly_time = 0.0000
trial division touched 99728 sieve locations out of 2064384000
1699 rels found: 1276 full + 423 from 10689 partial, (6822.74 rels/sec)

sieving required 7875 total polynomials (124 'A' polynomials)
trial division touched 99728 sieve locations out of 2064384000
total reports = 99728, total surviving reports = 32675
total blocks sieved = 63000, avg surviving reports per block = 0.52
Elapsed time: 1.7537 sec
elapsed time for ~10k relations of c65 = 1.7539 seconds.
extrapolated time for complete factorization = 4.8349 seconds
double free or corruption (!prev)
Aborted[/CODE]

If I do siqs(rsa()) to generate a random similarly sized (or larger) number, it does just fine.

kruoli 2022-09-22 06:30

[QUOTE=James Heinrich;613854]Xeon E5-2687W[/QUOTE]

It's E5-2587W [U]v4[/U], which is much newer.

James Heinrich 2022-09-22 13:53

[QUOTE=kruoli;613888]It's E5-2587W [U]v4[/U], which is much newer.[/QUOTE]Oops. :redface:
I keep a list of nearly 5000 CPUs for Prime95 benchmark data, and yet I missed that. :richd:

kruoli 2022-09-22 14:05

One has to admit that this naming scheme was not very evident; the one for Intel Core i is much more understandable (excluding some oddities like recent mobile CPUs etc.).

bsquared 2022-09-22 14:45

Problem in tune() fixed.
Also fixed an issue in microecm when compiling with gcc (verified it works now with gcc-11.1.0).
Re-built and re-uploaded the windows executables.

James Heinrich 2022-09-22 16:08

1 Attachment(s)
[QUOTE=bsquared;613908]Problem in tune() fixed.[/QUOTE]New build of regular version completed tune fine on my i3-8100, and it's still running on my i7-3930K.

I do notice that the 3930K is [i]embarrassingly[/i] slow in some cases, perhaps there's a fallback codepath due to lack of hardware features.
For example, SIQS on c90 [c]42735....7841[/c] ran at 1135 rel/s on my i3, and is currently running at 3.26 on my 3930K. That's 350x slower, is that sane and/or expected?

bsquared 2022-09-22 16:28

[QUOTE=James Heinrich;613916]New build of regular version completed tune fine on my i3-8100, and it's still running on my i7-3930K.

I do notice that the 3930K is [i]embarrassingly[/i] slow in some cases, perhaps there's a fallback codepath due to lack of hardware features.
For example, SIQS on c90 [c]42735....7841[/c] ran at 1135 rel/s on my i3, and is currently running at 3.26 on my 3930K. That's 350x slower, is that sane and/or expected?[/QUOTE]

No, that's not right. Thanks, I'll take a look.

wombatman 2022-09-22 20:33

[QUOTE=bsquared;613908]Problem in tune() fixed.
Also fixed an issue in microecm when compiling with gcc (verified it works now with gcc-11.1.0).
Re-built and re-uploaded the windows executables.[/QUOTE]

Just pulled the latest and can confirm tune() works flawlessly now. Thanks! :smile:

kotenok2000 2022-09-23 14:58

When i run
yafu.exe "factor(3105695207255595953041248693082694537249993263358883561705428359401428617)" -p -threads 8
with use_gpuecm yafu runs siqs with one thread.

kotenok2000 2022-09-23 15:32

GNFS also runs with one thread. Even with lathreads=8

storm5510 2022-09-23 15:58

[QUOTE=kotenok2000;613969]When i run
yafu.exe "factor(3105695207255595953041248693082694537249993263358883561705428359401428617)" -p -threads 8
with use_gpuecm yafu runs siqs with one thread.[/QUOTE]

It wants to see the thread value in [I]yafu.ini[/I]. I have mine set to 6 and it uses all 6. If you have not ran "tune" then you should.

As far as I know, YAFU has no GPU capability.

kotenok2000 2022-09-23 16:02

At post 440 bsquared says that yafu supports gpu ecm in 2.10
[url]https://www.mersenneforum.org/showpost.php?p=613759&postcount=440[/url]

James Heinrich 2022-09-23 16:15

[QUOTE=bsquared;613759]Almost forgot... external GPU-ECM binaries are now supported, optionally with Seth Trosi's CGBN enhancements. new options: "gpucurves", "cgbn", "use_gpuecm", "use_gpudev". Note that if you are using GPU-ECM with "factor()", then you'll probably want to make use of the "ext_ecm" option. This option sets when to switch over from internal non-gpu ECM to the external GPU-ECM binary. Otherwise, for example, factor() will try to use potentially thousands of GPU curves at B1=50000 when it only needs a few hundred. If you have AVX-ECM, then that will still be used as the internal ECM version before switching to external GPU-ECM.[/QUOTE]I saw that mentioned in the changelog. Perhaps someone who knows more about it can point us in the right direction for finding out how to use said "GPU ECM", including where to find Windows binaries if they exist. Does it support both NVIDIA and AMD?

kotenok2000 2022-09-23 16:18

[url]https://download.mersenne.ca/GMP-ECM/ecm704dev-svn2990-win64-cuda.zip[/url]

James Heinrich 2022-09-23 16:26

[QUOTE=kotenok2000;613977][url]https://download.mersenne.ca/GMP-ECM/[/url][/QUOTE]Ah, yes, well, um... :redface:

Appears to be CUDA-only so I can't play around with it. :no:

kotenok2000 2022-09-23 16:27

You will need to get cudart64_80.dll file
[url]https://download.mersenne.ca/CUDA-DLLs/CUDA-8.0[/url]

bsquared 2022-09-23 17:19

[QUOTE=kotenok2000;613969]When i run
yafu.exe "factor(3105695207255595953041248693082694537249993263358883561705428359401428617)" -p -threads 8
with use_gpuecm yafu runs siqs with one thread.[/QUOTE]

Did you Ctrl-C out of the gpu-ecm run? Threads is temporarily set to 1 to run gpu-ecm, but it should be set back to whatever it was before. A Ctrl-C might prevent it being set back.

bsquared 2022-09-23 17:23

[QUOTE=James Heinrich;613976]I saw that mentioned in the changelog. Perhaps someone who knows more about it can point us in the right direction for finding out how to use said "GPU ECM", including where to find Windows binaries if they exist. Does it support both NVIDIA and AMD?[/QUOTE]

[URL="https://www.mersenneforum.org/showthread.php?t=16480"]Here's [/URL]a thread for GPU-ECM.

And [URL="https://www.mersenneforum.org/showthread.php?t=27103"]here's [/URL]a thread for Seth Trosi's improved CGBN variant of GPU-ECM.

Both are available as compile options in [URL="https://gitlab.inria.fr/zimmerma/ecm"]gmp-ecm[/URL]. Only CUDA is supported.

kotenok2000 2022-09-23 17:24

2 Attachment(s)
I use this perl script with Strawberry Perl

storm5510 2022-09-24 16:12

[QUOTE=kotenok2000;613977][url]https://download.mersenne.ca/GMP-ECM/ecm704dev-svn2990-win64-cuda.zip[/url][/QUOTE]

No sale! This does not use a GPU, at least, not mine, RTX 2080. Yes, I added the needed switches... :no:

kotenok2000 2022-09-24 16:44

Does it crash?

James Heinrich 2022-09-24 16:58

[QUOTE=storm5510;614030]This does not use a GPU, at least, not mine[/QUOTE]Perhaps some copy-paste of YAFU output and you .ini file might help track down the problem.

kotenok2000 2022-09-24 17:00

Try to change
[CODE]% crossover to use the external ECM executable over the internally linked
% ECM (or avx-ecm)
ext_ecm=1000000000[/CODE]
to
[CODE]% crossover to use the external ECM executable over the internally linked
% ECM (or avx-ecm)
ext_ecm=10000[/CODE]
And uncomment prefer_gmpecm
in yafu.ini

James Heinrich 2022-09-24 20:08

I spent the last 2 hours trying to track down an off-by-one error in my own code, only to find there's a bug in YAFU v2.10 (and probably other versions) handling of [c]batchfile[/c] inputs.

[b]bug:[/b] YAFU ignores the last line in a [c]batchfile[/c] input file if the last line doesn't have a linebreak at the end. What makes this bad is that it not only doesn't do the last assignment, but it thinks it's done everything and erases the batchfile input file, deleting the last assignment line:
[c]eof; done processing batchfile[/c]

Adding a linebreak after the last assignment fixes the problem for now, but this is unexpected behavior.


All times are UTC. The time now is 20:00.

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2022, Jelsoft Enterprises Ltd.