mersenneforum.org > YAFU AVX-ECM
 Register FAQ Search Today's Posts Mark Forums Read

2020-10-30, 21:08   #45
mathwiz

Mar 2019

11·13 Posts

Quote:
 Originally Posted by bsquared I've checked the latest code into r393 if anyone wants to do some testing. You will need a avx512 capable cpu.
Is https://github.com/bbuhrow/avx-ecm/ the latest repo? Unless I'm reading wrong, the only commit recently (3 days ago) was just to README.md?

2020-10-30, 21:13   #46
bsquared

"Ben"
Feb 2007

336410 Posts

Quote:
 Originally Posted by mathwiz Is https://github.com/bbuhrow/avx-ecm/ the latest repo? Unless I'm reading wrong, the only commit recently (3 days ago) was just to README.md?
Ah, good question. Sorry, I've folded it all into yafu, so it is yafu r393 that you'll need. It is probably a good idea to update the avx-ecm repo as well since that is significantly easier to build. I can do that tonight, hopefully.

 2020-10-30, 22:14 #47 VBCurtis     "Curtis" Feb 2005 Riverside, CA 11·421 Posts One more reason to buy one of those Phi barebones systems! Nice work.
2020-10-30, 22:50   #48
mathwiz

Mar 2019

11×13 Posts

Quote:
 Originally Posted by bsquared Ah, good question. Sorry, I've folded it all into yafu, so it is yafu r393 that you'll need. It is probably a good idea to update the avx-ecm repo as well since that is significantly easier to build. I can do that tonight, hopefully.
Gotcha. Is there an easy way to build just AVX-ECM from the latest yafu sources? Or, if you could update the avx-ecm git repo, that'd be great too!

I tried:

Code:
$svn checkout https://svn.code.sf.net/p/yafu/code/branches/wip yafu-code$ make -j 8 SKYLAKEX=1
But I get some linker errors like:

Code:
gcc-9: error: /sppdg/scratch/buhrow/projects/gmp_install/gmp-6.2.0/lib/libgmp.a: No such file or directory

2020-10-31, 00:36   #49
bsquared

"Ben"
Feb 2007

22×292 Posts

Quote:
 Originally Posted by VBCurtis One more reason to buy one of those Phi barebones systems! Nice work.
Thanks!

I actually have some data for the KNL, linearly scaled from a run with B1=7e6.

7210 KNL:
2048 simultaneous curves with 256 threads
354 hours/run
10.4 minutes/curve

7250 KNL
2176 simultaneous curves with 272 threads
353 hours/run
9.7 minutes/curve

B2 then takes about 3/4 of that to go up to 100*B1, another 264 hours, so that's a lengthy ~25 day runtime per batch. But you get 2000+ curves at the end of it (B1=7G).

Running with fewer threads and curves trades off throughput for lower wall-clock time, if that is a concern. I don't have data for the exact trades there, yet. Also, it will save intermediate checkpoint files every 100M primes for each curve. The savefiles can be fed to GMP-ECM for stage 2, if you'd rather run stage 2 that way. Run with -saveB1 on the command line to enable saving.

Memory use is pretty minimal for both stages; won't be a problem to stay in MCDRAM.

Last fiddled with by bsquared on 2020-10-31 at 00:56

2020-10-31, 00:50   #50
bsquared

"Ben"
Feb 2007

22×292 Posts

Quote:
 Originally Posted by mathwiz Gotcha. Is there an easy way to build just AVX-ECM from the latest yafu sources? Or, if you could update the avx-ecm git repo, that'd be great too! I tried: Code: $svn checkout https://svn.code.sf.net/p/yafu/code/branches/wip yafu-code$ make -j 8 SKYLAKEX=1 But I get some linker errors like:
No easy way to just build part of yafu. Makefiles are not my strong suit. I very much recommend EdH's guides on building yafu. It requires a small amount of makefile editing to point to your msieve, ecm, and gmp locations.

IMO there are reasons to prefer the yafu version of avx-ecm. It integrates into factor(), so it's fire and forget on any input. It will auto-recognize Mersennes and determine the fastest approach (sometimes it's better to just use Montgomery arithmetic). It uses the NFS algebraic factor extraction framework that's already there, so you could feed it, say, (2^1265-1) / 121441 / 533831 / 136149590957161 / 556590676780973231 and it would pull out all the annoying algebraic factors and go to work on the remaining C223. You can tell it to pretest up to a specific t-level, etc.

avx-ecm has none of that, but I will update it with the new vector math at least, with maybe a command line switch to turn it on.

 2020-11-01, 04:38 #51 wombatman I moo ablest echo power!     May 2013 1,741 Posts Posting here only because the issue seems to be related to avx-ecm. I'm trying to build the latest revision of YAFU, and no matter what I pass during make, it's failing with errors like: Code: /usr/bin/ld: factor/avx-ecm/avx_ecm_main.o: in function vec_ecm_main': /home/wombat/yafu-code/branches/wip/factor/avx-ecm/avx_ecm_main.c:266: undefined reference to snfs_init' /usr/bin/ld: /home/wombat/yafu-code/branches/wip/factor/avx-ecm/avx_ecm_main.c:272: undefined reference to find_primitive_factor' /usr/bin/ld: /home/wombat/yafu-code/branches/wip/factor/avx-ecm/avx_ecm_main.c:279: undefined reference to snfs_clear' /usr/bin/ld: /home/wombat/yafu-code/branches/wip/factor/avx-ecm/avx_ecm_main.c:464: undefined reference to vecmulmod52_mersenne' /usr/bin/ld: /home/wombat/yafu-code/branches/wip/factor/avx-ecm/avx_ecm_main.c:465: undefined reference to vecsqrmod52_mersenne' collect2: error: ld returned 1 exit status make: *** [Makefile:397: all] Error 1 I'm passing in this: Code:  make clean && make ECM=1 USE_SSE41=1 This error is recent, though I can't say exactly which revision I was able to successfully build. Any suggestions?
2020-11-01, 15:10   #52
bsquared

"Ben"
Feb 2007

22·292 Posts

Quote:
 Originally Posted by wombatman Posting here only because the issue seems to be related to avx-ecm. I'm trying to build the latest revision of YAFU, and no matter what I pass during make, it's failing with errors like: Code: /usr/bin/ld: factor/avx-ecm/avx_ecm_main.o: in function vec_ecm_main': /home/wombat/yafu-code/branches/wip/factor/avx-ecm/avx_ecm_main.c:266: undefined reference to snfs_init' /usr/bin/ld: /home/wombat/yafu-code/branches/wip/factor/avx-ecm/avx_ecm_main.c:272: undefined reference to find_primitive_factor' /usr/bin/ld: /home/wombat/yafu-code/branches/wip/factor/avx-ecm/avx_ecm_main.c:279: undefined reference to snfs_clear' /usr/bin/ld: /home/wombat/yafu-code/branches/wip/factor/avx-ecm/avx_ecm_main.c:464: undefined reference to vecmulmod52_mersenne' /usr/bin/ld: /home/wombat/yafu-code/branches/wip/factor/avx-ecm/avx_ecm_main.c:465: undefined reference to vecsqrmod52_mersenne' collect2: error: ld returned 1 exit status make: *** [Makefile:397: all] Error 1 I'm passing in this: Code:  make clean && make ECM=1 USE_SSE41=1 This error is recent, though I can't say exactly which revision I was able to successfully build. Any suggestions?
Looks like you just need to use more makefile options. You need at least NFS=1 to bring in the nfs stuff and SKYLAKEX=1 for the avx512 vector arithmetic stuff. And if you use SKYLAKEX=1 then you also need USE_AVX2=1. My typical build line looks like this:

make NFS=1 USE_AVX2=1 USE_BMI2=1 SKYLAKEX=1

also, ECM=1 is a msieve thing, not a yafu thing, so you can leave that off.

 2020-11-01, 15:30 #53 bsquared     "Ben" Feb 2007 22·292 Posts Another quick usage note... Normally, yafu will switch over from using an internal GMP-ECM to an external GMP-ECM (pointed to by the ecm_path variable in yafu.ini) at B1=50k. With AVX-ECM enabled, by default that switchover happens at B1=40M instead. If you want to use AVX-ECM at higher B1, put that crossover value into the variable "ext_ecm" in yafu.ini. For example if I want to use B1=110M with AVX-ECM and not with the external GMP-ECM, set ext_ecm=120000000, or anything larger than 110M.
 2020-11-01, 15:30 #54 mathwiz   Mar 2019 11·13 Posts (waits patiently for the old avx-ecm git repo to be updated with the latest code... it's so much easier to build :)
2020-11-01, 16:56   #55
bsquared

"Ben"
Feb 2007

22·292 Posts

Quote:
 Originally Posted by mathwiz (waits patiently for the old avx-ecm git repo to be updated with the latest code... it's so much easier to build :)

Now updated, tested, seems to work. Also checked in a pre-built version for windows. The mingw compiler doesn't do as good of a job, but something for windows is better than nothing.