YAFU 2.0

[Brian Gladman]

Originally Posted by bsquared

I think CUDA_ERROR_FILE_NOT_FOUND refers to the stage1_core_sm**.ptx file; it is supposed to be next to the executable. The sort_engine.dll file should be in a directory called cub beneath the executable.

Here are my files from a working msieve (as a zip file):

https://drive.google.com/file/d/1055...ew?usp=sharing

[Plutie]

Having issues building w/ AVX512 support.



This command, which seems like it should work by itself from a glance at the Makefile, is exiting with many errors.

make ICELAKE=1 -j72

./libysiqs.a(SIQS.o): In function `process_poly':
/root/yafu/factor/qs/SIQS.c:1282: undefined reference to `nextRoots_32k_avx2_small'
./libysiqs.a(SIQS.o): In function `siqs_static_init':
/root/yafu/factor/qs/SIQS.c:2153: undefined reference to `tdiv_medprimes_32k_avx2'
/root/yafu/factor/qs/SIQS.c:2164: undefined reference to `resieve_medprimes_32k_avx512bw'
/root/yafu/factor/qs/SIQS.c:2165: undefined reference to `med_sieveblock_32k_avx512bw'
/root/yafu/factor/qs/SIQS.c:2209: undefined reference to `resieve_medprimes_32k_avx2'
/root/yafu/factor/qs/SIQS.c:2210: undefined reference to `tdiv_medprimes_32k_avx2'
/root/yafu/factor/qs/SIQS.c:2212: undefined reference to `med_sieveblock_32k_avx2'
/root/yafu/factor/qs/SIQS.c:2241: undefined reference to `nextRoots_32k_avx2_intrin'
/root/yafu/factor/qs/SIQS.c:2256: undefined reference to `nextRoots_32k_sse41'
/root/yafu/factor/qs/SIQS.c:2257: undefined reference to `med_sieveblock_32k_sse41'
/root/yafu/factor/qs/SIQS.c:2164: undefined reference to `resieve_medprimes_32k_avx512bw'
/root/yafu/factor/qs/SIQS.c:2165: undefined reference to `med_sieveblock_32k_avx512bw'
/root/yafu/factor/qs/SIQS.c:2176: undefined reference to `resieve_medprimes_32k_avx2'
/root/yafu/factor/qs/SIQS.c:2178: undefined reference to `med_sieveblock_32k_avx2'
/root/yafu/factor/qs/SIQS.c:2249: undefined reference to `nextRoots_32k_avx2'
/root/yafu/factor/qs/SIQS.c:2176: undefined reference to `resieve_medprimes_32k_avx2'
/root/yafu/factor/qs/SIQS.c:2178: undefined reference to `med_sieveblock_32k_avx2'
collect2: error: ld returned 1 exit status
Makefile:415: recipe for target 'yafu' failed
make: *** [yafu] Error 1

Running make with these arguments has fixed most of the undefined references, but some still remain.

make ICELAKE=1 SKYLAKEX=1 USE_AVX2=1 USE_BMI2=1 NFS=1 -j72

./libysiqs.a(SIQS.o): In function `siqs_static_init':
/root/yafu/factor/qs/SIQS.c:2164: undefined reference to `resieve_medprimes_32k_avx512bw'
/root/yafu/factor/qs/SIQS.c:2165: undefined reference to `med_sieveblock_32k_avx512bw'
/root/yafu/factor/qs/SIQS.c:2164: undefined reference to `resieve_medprimes_32k_avx512bw'
/root/yafu/factor/qs/SIQS.c:2165: undefined reference to `med_sieveblock_32k_avx512bw'
collect2: error: ld returned 1 exit status
Makefile:415: recipe for target 'yafu' failed
make: *** [yafu] Error 1

Any help would be greatly appreciated, thank you!

[bsquared]

Originally Posted by Plutie

Having issues building w/ AVX512 support.


Running make with these arguments has fixed most of the undefined references, but some still remain.

make ICELAKE=1 SKYLAKEX=1 USE_AVX2=1 USE_BMI2=1 NFS=1 -j72

./libysiqs.a(SIQS.o): In function `siqs_static_init':
/root/yafu/factor/qs/SIQS.c:2164: undefined reference to `resieve_medprimes_32k_avx512bw'
/root/yafu/factor/qs/SIQS.c:2165: undefined reference to `med_sieveblock_32k_avx512bw'
/root/yafu/factor/qs/SIQS.c:2164: undefined reference to `resieve_medprimes_32k_avx512bw'
/root/yafu/factor/qs/SIQS.c:2165: undefined reference to `med_sieveblock_32k_avx512bw'
collect2: error: ld returned 1 exit status
Makefile:415: recipe for target 'yafu' failed
make: *** [yafu] Error 1

Any help would be greatly appreciated, thank you!

Can you verify that when building files and linking you actually see these flags being passed:

gcc -g -DUSE_SSE2 -DUSE_BMI2 -DUSE_AVX2 -DUSE_AVX512F -DUSE_AVX512BW -DSKYLAKEX -DIFMA -march=icelake-client ...

Particularly, the -DUSE_AVX512BW flag.

Actually, I think you just have to do a clean with the flags specified:

make clean ICELAKE=1 SKYLAKEX=1 USE_AVX2=1 USE_BMI2=1 NFS=1

then remake. If the relevant file (med_sieve_32k_avx2.c) was compiled before without -DUSE_AVX512BW then make won't re-build it just because a new flag is specified, the old object file first has to be removed (or source file touched).

[Plutie]

Originally Posted by bsquared

Can you verify that when building files and linking you actually see these flags being passed:

gcc -g -DUSE_SSE2 -DUSE_BMI2 -DUSE_AVX2 -DUSE_AVX512F -DUSE_AVX512BW -DSKYLAKEX -DIFMA -march=icelake-client ...

Particularly, the -DUSE_AVX512BW flag.

Actually, I think you just have to do a clean with the flags specified:

make clean ICELAKE=1 SKYLAKEX=1 USE_AVX2=1 USE_BMI2=1 NFS=1

then remake. If the relevant file (med_sieve_32k_avx2.c) was compiled before without -DUSE_AVX512BW then make won't re-build it just because a new flag is specified, the old object file first has to be removed (or source file touched).

Sorry for the delay. Tried that, still isn't working. (also realized that the CPU in this system isn't actually ice lake, but skylake-x). Tried rebuilding after the clean with

make SKYLAKEX=1 NFS=1

This didn't work either, and here are the flags being passed to gcc.

gcc-8 -g -DUSE_SSE2 -DUSE_BMI2 -DUSE_AVX2 -DUSE_AVX512F -DUSE_AVX512BW -DSKYLAKEX

[bsquared]

Originally Posted by Plutie

Sorry for the delay. Tried that, still isn't working. (also realized that the CPU in this system isn't actually ice lake, but skylake-x). Tried rebuilding after the clean with

make SKYLAKEX=1 NFS=1

This didn't work either, and here are the flags being passed to gcc.

gcc-8 -g -DUSE_SSE2 -DUSE_BMI2 -DUSE_AVX2 -DUSE_AVX512F -DUSE_AVX512BW -DSKYLAKEX

It may seem redundant, but after a clean try this:

make SKYLAKEX=1 USE_AVX2=1 USE_BMI2=1 NFS=1

That is the build line I use all the time; if it doesn't work you can post or PM me the errors. I am working to make builds easier, thanks for your patience.

[Plutie]

That worked, thank you!

[chris2be8]

Originally Posted by bsquared

I think CUDA_ERROR_FILE_NOT_FOUND refers to the stage1_core_sm**.ptx file; it is supposed to be next to the executable. The sort_engine.dll file should be in a directory called cub beneath the executable.

See https://mersenneforum.org/showpost.p...8&postcount=50 for details of what I found out about the latest msieve version.

[stream]

Pure AVX2 build is broken:

make yafu COMPILER=gcc USE_AVX2=1 USE_BMI2=1

(NFS is intentionally disabled)

Linking fails with bunch of references to AVX-512 code.

./libyecm.a(avx_ecm_main.o): In function `vec_ecm_main':
/home/stream/work/yafu/yafu/factor/avx-ecm/avx_ecm_main.c:585: undefined reference to `vecmulmod52_fixed832_bfips'
/home/stream/work/yafu/yafu/factor/avx-ecm/avx_ecm_main.c:586: undefined reference to `vecsqrmod52_fixed832_bfips'
/home/stream/work/yafu/yafu/factor/avx-ecm/avx_ecm_main.c:587: undefined reference to `vec_simul_addsub52_fixed1040'
/home/stream/work/yafu/yafu/factor/avx-ecm/avx_ecm_main.c:578: undefined reference to `vecmulmod52_fixed624_bfips'
/home/stream/work/yafu/yafu/factor/avx-ecm/avx_ecm_main.c:579: undefined reference to `vecsqrmod52_fixed624_bfips'
/home/stream/work/yafu/yafu/factor/avx-ecm/avx_ecm_main.c:580: undefined reference to `vec_simul_addsub52_fixed1040'
/home/stream/work/yafu/yafu/factor/avx-ecm/avx_ecm_main.c:571: undefined reference to `vecmulmod52_fixed416_bfips'
/home/stream/work/yafu/yafu/factor/avx-ecm/avx_ecm_main.c:572: undefined reference to `vecsqrmod52_fixed416_bfips'
/home/stream/work/yafu/yafu/factor/avx-ecm/avx_ecm_main.c:573: undefined reference to `vec_simul_addsub52_fixed1040'
/home/stream/work/yafu/yafu/factor/avx-ecm/avx_ecm_main.c:592: undefined reference to `vecmulmod52_fixed1040_bfips'
/home/stream/work/yafu/yafu/factor/avx-ecm/avx_ecm_main.c:593: undefined reference to `vecsqrmod52_fixed1040_bfips'
/home/stream/work/yafu/yafu/factor/avx-ecm/avx_ecm_main.c:594: undefined reference to `vec_simul_addsub52_fixed1040'

[johnadam74]

Revision 536 will compile but something must have changed in Revision 537 to break the code?

[James Heinrich]

I'm back home now and able to test v2.06 on my Win10/i8-8100 machine.
It works, but I've noticed a couple things (which entirely could be me doing something wrong).

1) ECM runs as a single thread of yafu-x64.exe
I don't care if ECM runs within yafu or spawns external ecm.exe as long as it uses the appropriate number of threads.
Relevant yafu.ini entries:
threads=4
ecm_path=c:\users\user\desktop\yafu2\ecm\ecm.exe
If I specify an invalid path for ecm.exe then it appropriately tells me "ecm: ECM executable does not exist at c:\invalid\path\ecm.exe" and "using internal single threaded ECM...", but even when the path to ecm.exe is valid it appears to be using the internal, single-threaded ECM (with no warning).

2) Number of witnesses for PRP is always "1" no even if I set something like nprp=20 in yafu.ini

3) Increased verbosity doesn't seem to work (at least in yafu.ini). The description says "Note that more v's increase the verbosity" but if I specify v v it tells me "invalid option v v" and spews the list of valid options and exits

[bsquared]

Originally Posted by stream

Pure AVX2 build is broken:

make yafu COMPILER=gcc USE_AVX2=1 USE_BMI2=1

(NFS is intentionally disabled)

Linking fails with bunch of references to AVX-512 code.

./libyecm.a(avx_ecm_main.o): In function `vec_ecm_main':
/home/stream/work/yafu/yafu/factor/avx-ecm/avx_ecm_main.c:585: undefined reference to `vecmulmod52_fixed832_bfips'
/home/stream/work/yafu/yafu/factor/avx-ecm/avx_ecm_main.c:586: undefined reference to `vecsqrmod52_fixed832_bfips'
/home/stream/work/yafu/yafu/factor/avx-ecm/avx_ecm_main.c:587: undefined reference to `vec_simul_addsub52_fixed1040'
/home/stream/work/yafu/yafu/factor/avx-ecm/avx_ecm_main.c:578: undefined reference to `vecmulmod52_fixed624_bfips'
/home/stream/work/yafu/yafu/factor/avx-ecm/avx_ecm_main.c:579: undefined reference to `vecsqrmod52_fixed624_bfips'
/home/stream/work/yafu/yafu/factor/avx-ecm/avx_ecm_main.c:580: undefined reference to `vec_simul_addsub52_fixed1040'
/home/stream/work/yafu/yafu/factor/avx-ecm/avx_ecm_main.c:571: undefined reference to `vecmulmod52_fixed416_bfips'
/home/stream/work/yafu/yafu/factor/avx-ecm/avx_ecm_main.c:572: undefined reference to `vecsqrmod52_fixed416_bfips'
/home/stream/work/yafu/yafu/factor/avx-ecm/avx_ecm_main.c:573: undefined reference to `vec_simul_addsub52_fixed1040'
/home/stream/work/yafu/yafu/factor/avx-ecm/avx_ecm_main.c:592: undefined reference to `vecmulmod52_fixed1040_bfips'
/home/stream/work/yafu/yafu/factor/avx-ecm/avx_ecm_main.c:593: undefined reference to `vecsqrmod52_fixed1040_bfips'
/home/stream/work/yafu/yafu/factor/avx-ecm/avx_ecm_main.c:594: undefined reference to `vec_simul_addsub52_fixed1040'

Thanks, it's fixed now.

Those new function make avx-ecm about 10% faster below 1040 bits, but I had forgotten to add stubs for non-avx-512 builds.