mersenneforum.org > YAFU ggnfs improvements
 Register FAQ Search Today's Posts Mark Forums Read

2022-10-21, 14:22   #67
henryzz
Just call me Henry

"David"
Sep 2007
Liverpool (GMT/BST)

37×163 Posts

Quote:
 Originally Posted by bsquared More frustrations. Trying to make the f-versions of the sievers fails because input-poly-orig doesn't exist (none of .w .c or .h). But there is an input-poly.orig in the lasieve4 source! (note the .orig instead of -orig in the name). Ok, copy over input-poly.orig.c, rename it to input-poly-orig.c, create an input-poly-orig.h which has a declaration for the input_poly_orig function and attempt to make allf. And it works! But, the sievers won't run with the error: Code: Cannot read number which is to be factored: Success Attempt 2: Rename input-poly.c and input-poly.h, which do exist in the repo, to input-poly-orig.c and input-poly-orig.h. Build works again! But, the sievers won't run with the error: Code: Input line = Rational sieve parameters like sievemin FBbound sieverest_multiplier max_prime_bits max_factorbits eg. 20 1e6 3.2 30 50 lacking from input file nfs300_650k_I11.job Anyone know how to obtain a correct copy of input-poly-orig.c and input-poly-orig.h so that the f- and g-versions of the siever can be built?
The f and g versions probably don't have many of the ggnfs modifications although there have been working versions of the f siever around in the past. The f version allows composite special q and sieving special q below the fb limits. Sieving only prime special q it was a slightly slower. Composite special q tend to increase the duplication rate. There may be situations where this is useful to improve overall yield on very large jobs.
The g version looks like it adds batch trial division. I don't know whether this is an improvement.

I think I did get both f and g to compile at one point(maybe 10 years ago). I am not sure that I got g working. Will have a look at what I have from the distant past tomorrow.
I remember f was a bit of a fiddle to get working.

Last fiddled with by henryzz on 2022-10-21 at 14:22

 2022-10-21, 15:26 #68 wreck     "Bo Chen" Oct 2005 Wuhan,China 3×61 Posts I have done the sieve from 316M to 317M of R1340L, 959668 relations collected, while 315M to 316M collect 1291314 relations, about 331646 less relations. 316000000 to 316492661 sieved using original gnfs binary under windows 10 (628194 relations, 1.27 rel/q), while 316492661 to 317000000 sieved using avx512-gnfs under Ubuntu 20.04 (331474 relations, 0.653 rel/q; if remove the q from 316724521 to 316963841, then 268019 special-Q, 1.23 rel/q, about the same as original ggnfs). I dig to found that no q between 316724521 to 316963841, there exist five relations in the relation file. Code: -279653911,270203771:2C47EC05,5455,5117,D955FD7,792367,6CA4A7:26D57BED,8D97BD69,E1EA112D,655,FBBBA0D,19C8371,13541B1,C33305,12E0D529 355978387,122689099:821,570B,1EC7,882D177,10E0F7,4467B,8C5D:1DFEA7DD,96D50931,8AD,B5D2F55,594CBB9,FF9539,8E8D85,528D49,20E55,12E0D529 -128782835,236497531:1048C1371,11395B29,4DC2601,415FAB,1C0C9:-1440211,431542:10AA4D87,66A862F,388838F,348CCD,11919D:F69C5DDD,1C5CBE439,2FF9,655,3E470D,11DFDD,12E47C01 12213311,17980164:DF3,3F1,1D71,1CFC1539,17BA9CD1,30383F:13BBD45D,63D9,26D5,565,267AF3D,13D092D,8CF335,5904D,12E47C01 3312109,19371270:99E70D45,1996DCB,5EBC01,31D3B3,F8CB:22FE5F21,DBC93D,2268AD,1CC9ED,1584FD,71035,99AD,12E47C01 I re-sieve q from 316724521 to 316724621, the relations are correct, the right five relations are Code: -279653911,270203771:2C47EC05,5455,5117,D955FD7,792367,6CA4A7:26D57BED,8D97BD69,E1EA112D,655,FBBBA0D,19C8371,13541B1,C33305,12E0D529 355978387,122689099:821,570B,1EC7,882D177,10E0F7,4467B,8C5D:1DFEA7DD,96D50931,8AD,B5D2F55,594CBB9,FF9539,8E8D85,528D49,20E55,12E0D529 -128782835,236497531:1048C1371,11395B29,4DC2601,415FAB,1C0C9:884E1065,46C9,BB9,D2B0A41,C501139,21E218D,4460B1,20D781,35071,12E0D529 -81467213,471237169:577,7E24763,2028C6B,ABCFFD,56F7B,195EF:16F2EA5D5,88B50671,8E9,5281,DE6A729,A34815,5FAD49,1717A5,C289,12E0D529 78951107,56007340:40D52CC5,73CA173,287E079,601BBF,9BEC7:16A53A29,CED58865,7BFD,2A1D,2635,5CA4235,26D80B1,14062D,12E0D529 I try to reproduce the problem with file R1340L_16e_a_316500000_317000000.out's content is Code: -279653911,270203771:2C47EC05,5455,5117,D955FD7,792367,6CA4A7:26D57BED,8D97BD69,E1EA112D,655,FBBBA0D,19C8371,13541B1,C33305,12E0D529 355978387,122689099:821,570B,1EC7,882D177,10E0F7,4467B,8C5D:1DFEA7DD,96D50931,8AD,B5D2F55,594CBB9,FF9539,8E8D85,528D49,20E55,12E0D529 -128782835,236497531:1048C1371,11395B29,4DC2601,415FAB,1C0C9: And with the command ./gnfs-lasieve4I16e -v -f 316500000 -c 500000 -o R1340L_16e_a_316500000_317000000.out -a R1340L_poly.txt -R But the relations seems correct this time, Code: -279653911,270203771:2C47EC05,5455,5117,D955FD7,792367,6CA4A7:26D57BED,8D97BD69,E1EA112D,655,FBBBA0D,19C8371,13541B1,C33305,12E0D529 355978387,122689099:821,570B,1EC7,882D177,10E0F7,4467B,8C5D:1DFEA7DD,96D50931,8AD,B5D2F55,594CBB9,FF9539,8E8D85,528D49,20E55,12E0D529 -128782835,236497531:1048C1371,11395B29,4DC2601,415FAB,1C0C9: 2772363,13511359:287F6A29,E3B69429,4EB67B,56DC7,F9FD:BA04BBF9,4DB5FFE9,EB1,216B0C9,48A2D9,FFF29,12E0D529 15943849,3724237:689CBD57,39E5,2F27,6739B8F,2C9EF9,80BD:78892B29,C8B44AB1,85D,1069,47DE495,1A010A9,1C14D,9B8D,12E0D529 -971678175,1191188861:8B5A9FFF,18600551F,D401D,6C4D3,F265:12E965A1D,7A504FF5,E76C0A75,C030961,77A02A5,42681B9,185999,1413C5,12E0D529 The original command I use is ./gnfs-lasieve4I16e -v -f 316000000 -c 1000000 -o R1340L_16e_a_316000000_317000000.out -a R1340L_poly.txt -R I write this command as a bash file #!/bin/bash let StartQ=316000000 let SieveQ=1000000 let EndQ=StartQ+SieveQ echo "./gnfs-lasieve4I16e -v -f $StartQ -c$SieveQ -o R1340L_16e_a_${StartQ}_${EndQ}.out -a R1340L_poly.txt -R" ./gnfs-lasieve4I16e -v -f $StartQ -c$SieveQ -o R1340L_16e_a_${StartQ}_${EndQ}.out -a R1340L_poly.txt -R I open and close the dos's bat or Linux's bash file about 50 times (there is a time that I open the bash file twice). Under Windows I use Control + C, it will ask me whether I want to close the window, I type y, And the next time I open it, there is no complain. Under Linux I use Control + C or Control + Z, the programe close immediately, and when next time I open it, it complains another new line ' Warning: an incomplete line in the original file; if just a few, it's ok, they will be skipped'. Not sure what happened here, perhaps a parse error. I'm ready to sieve q from 316724521 to 316963841 again.
2022-10-21, 18:00   #69
henryzz
Just call me Henry

"David"
Sep 2007
Liverpool (GMT/BST)

10111100011112 Posts

I am struggling to get the code I have to compile. I am getting many multiple definition errors for functions like asm_zero, asm_copy etc.

The strange thing is that I am getting these errors on code that I compiled in 2014.

I have found instructions for compiling on https://www.mersenneforum.org/showth...t=19711&page=4 post 64
The attachment link seems to be broken, so I have attached what I believe to be the referenced file. Can anyone else compile following these instructions?

Once I get it to compile, I believe I have an edited file for the 5f siever.
Attached Files
 lasieve5_smjsmod5.tar.bz2 (312.0 KB, 63 views)

2022-10-21, 18:36   #70
frmky

Jul 2003
So Cal

23×52×13 Posts

Quote:
 Originally Posted by henryzz I am getting many multiple definition errors for functions like asm_zero, asm_copy etc.
That's new in gcc 10 and higher. Use gcc 9 or lower. I haven't taken the time to track that down.

2022-10-21, 19:29   #71
henryzz
Just call me Henry

"David"
Sep 2007
Liverpool (GMT/BST)

178F16 Posts

Quote:
 Originally Posted by frmky That's new in gcc 10 and higher. Use gcc 9 or lower. I haven't taken the time to track that down.

Thanks for the hint.
I think it is something to do with montgomery_mul.h being referenced from multiple places. It seems to be in both strategy.o and liblasieve.a.

 2022-10-22, 14:53 #72 bsquared     "Ben" Feb 2007 3,733 Posts The same thing happened in yafu when I switched to gcc 11 from 9. It has to do with the function pointers "declared" in the header of montgomery_mul.h. I put that in quotes because I guess newer gcc's don't consider those declarations anymore. ICX also throws errors. ICC is fine with it. I'll have to look up how I fixed that in yafu; I hope it would work here too.
2022-10-23, 03:37   #73
SethTro

"Seth"
Apr 2019

7378 Posts

Quote:
 Originally Posted by bsquared For GGNFS that time is labeled MPQS and is reported when running with -v. Posts 34 and 48 show examples where this time is a noticeable fraction of the overall time. In those examples it was 8% and 3%, respectively, when using my new AVX512 vectorized ECM code. Without that it would be a little higher. When the large prime bounds are large and three large primes are allowed on one side or the other is when you'll run into this case. For smaller jobs that don't use 3LP, the MPQS (cofactorization) time is usually negligible (e.g, post 1 @ 420 bits where it is 0.2%). So it may indeed help on big jobs. The key will be finding out if device transfer overhead ruins the benefits or not. And also working the gpu into the overall 3LP factorization strategy, which has a number of steps. A stage 2 algorithm in the gpu is also probably a must. For the biggest of jobs, I've been thinking about ways to get Jasonp's Batch GCD method involved. I use it in yafu's 3LP variation for a really nice speedup.
Thank you for the detailed explanations in your responses.

It doesn't sound like it would be a big improvement at the current time. If at some future point this number is larger (15%? 25%?) It might be worth the work to extract the cuda code from gmp-ecm, make it support multiple numbers at the same time, and tune it to minimize setup overhead.

2022-10-23, 07:22   #74
Gimarel

Apr 2010

F816 Posts

Quote:
 Originally Posted by frmky That's new in gcc 10 and higher. Use gcc 9 or lower. I haven't taken the time to track that down.
Quote:
 Originally Posted by gcc-10 release notes GCC now defaults to -fno-common. As a result, global variable accesses are more efficient on various targets. In C, global variables with multiple tentative definitions now result in linker errors. With -fcommon such definitions are silently merged during linking
It seems that the option -fcommon should solve the problem.

2022-10-26, 02:38   #75
bsquared

"Ben"
Feb 2007

3,733 Posts

Quote:
 Originally Posted by Gimarel It seems that the option -fcommon should solve the problem.
This worked for me to build the e-versions using ICX, thank you!

lasieve5_64 is now checked in and has all of the same optimizations as lasieve4_64. Plus a couple more that provide maybe another few percent speedup.

What I should have done is make the changes to the cweb files in lasieve 5 instead of working with the ctangled code. That is a chore. I was porting from ctangled code in lasieve4, so that's what made sense at the time. I might circle back and see if I can do that.

Probably next on the agenda is seeing about a windows build, and doing some more testing.

 2022-10-27, 18:55 #76 bsquared     "Ben" Feb 2007 72258 Posts Fixed a silly bug that was preventing a small number of relations from being discovered. As a test I ran all special Q from 316000000 to 316100000 using wreck's poly for R1340L with the lasieve5_64 16e siever with the new code and the original code. Original: 123939 relations found in 174079 core seconds. New: 123689 relations found in 139200 core seconds. A 20% speed improvement at a cost of 250 (0.2%) missing relations. I verified that all relations from the new version are the same as from the original. All but 3 of the missing relations were 3LP's. The other 3 were 62+ bit 2LP's. These latter 3 could be picked at very low cost up by adding a backup method to the 2LP factorization strategy if uecm fails.
 2022-11-08, 16:57 #77 unconnected     May 2009 Moscow, Russia 11×263 Posts Hi, I've tried new binaries and got an error after several successful factorizations. For the reference - I use old good factMsieve.pl script which getting .poly-file or even number and do the rest Poly for c103 number: Code: $cat c103.poly n: 1525306632396272654256201398577639453116991431104195043037446924921873712961837838587765194333349697233 skew: 2272170.41 c0: 44470936714402386191207041425 c1: 64665054240226483277526 c2: 39035884984131219 c3: -20580193202 c4: 9240 Y0: -3584449133208765940001434 Y1: 16673014653679 rlim: 2330000 alim: 2330000 lpbr: 26 lpba: 26 mfbr: 52 mfba: 52 rlambda: 2.5 alambda: 2.5 type: gnfs qintsize: 50000 One of the job-files (these are intermediate files which used by the main script to sieve): Code: $ cat c103-2.job.T1 n: 1525306632396272654256201398577639453116991431104195043037446924921873712961837838587765194333349697233 m: c0: 44470936714402386191207041425 c1: 64665054240226483277526 c2: 39035884984131219 c3: -20580193202 c4: 9240 Y0: -3584449133208765940001434 Y1: 16673014653679 skew: 2272170.41 rlim: 2330000 alim: 1315000 lpbr: 26 lpba: 26 mfbr: 52 mfba: 52 rlambda: 2.5 alambda: 2.5 q0: 1315001 qintsize: 12499 #q1:1327500 Error: Code: $./gnfs-lasieve4I12e -k -o test.out -v -n0 -a c103-2.job.T1 gnfs-lasieve4I12e (with asm64,avx-512 mmx-td,lasetup,lasched,sieve1,ecm,tds0,search0,tdsched): L1_BITS=15 FBsize 100777+0 (deg 4), 171534+0 (deg 1) Sorted factor base on side 0: 1: 32495 2: 32397 Sorted factor base on side 1: 1: 168023 total yield: 56627, q=1319407 (0.00110 sec/rel; ETA 0h01m) td error : 5081 does not divide Aborted (core dumped) Error is also reproducible with lasieve4 binary (but on another job-file): Code: $ ./gnfs-lasieve4I12e -k -o test2.out -v -n0 -a c103-2.job.T2 gnfs-lasieve4I12e (with asm64,avx-512 mmx-td,lasetup,lasched,sieve1,ecm,tds0,search0,tdsched): L1_BITS=15 FBsize 93653+0 (deg 4), 171534+0 (deg 1) I,J: 11,10, n_i,n_j = 2048,1024, j_per_strip,n_strips=16,64 total yield: 67817, q=1233781 (0.00114 sec/rel; ETA 0h01m) td error : 5081 does not divide Aborted (core dumped) Last fiddled with by unconnected on 2022-11-08 at 17:27

 Similar Threads Thread Thread Starter Forum Replies Last Post chris2be8 YAFU 9 2022-02-17 17:52 nivek000 YAFU 1 2021-12-10 22:35 EdH YAFU 8 2018-03-14 17:22 Zeta-Flux Factoring 1 2007-08-07 22:40 ATH Factoring 3 2006-08-12 22:50

All times are UTC. The time now is 15:01.

Sun Feb 5 15:01:31 UTC 2023 up 171 days, 12:30, 1 user, load averages: 1.14, 0.84, 0.79