mersenneforum.org

mersenneforum.org (https://www.mersenneforum.org/index.php)
-   EdH (https://www.mersenneforum.org/forumdisplay.php?f=152)
-   -   How I install YAFU onto my Ubuntu Machines (https://www.mersenneforum.org/showthread.php?t=23087)

EdH 2018-02-24 01:53

How I install YAFU onto my Ubuntu Machines
 
(Note: I expect to keep the first post of each of these "How I Install..." threads up-to-date with the latest version. Please read the rest of each thread to see what may have led to the current set of instructions.)

This thread will explain the steps I use to install* YAFU onto a computer which is already running Ubuntu. This procedure should work for other linux distributions as well but the only other one I've currently tested is Debain.

*In this instance "install" is referring to the acquiring and compilation of the YAFU package only. The binary will have to be called using its respective path.

I will expect the user of these steps to be able to use the sudo command.

I will use a directory called Math for these installations. You may elect to replace "Math" with your own choice.

These instructions can be used as a reference only, if you want, but I will provide specifics that work for me. If you follow these steps as I provide them, you should end up with a working installation.

First, follow the procedures in:

[URL="http://www.mersenneforum.org/showthread.php?p=480597"] How I Install GMP onto my Ubuntu Machines[/URL]
,
[URL="http://www.mersenneforum.org/showthread.php?p=480608"] How I Install GMP-ECM onto my Ubuntu Machines[/URL]
,
[URL="http://www.mersenneforum.org/showthread.php?t=23085"]How I Install msieve onto my Ubuntu Machines[/URL]
and
[URL="https://www.mersenneforum.org/showthread.php?t=23081"]How I Install ggnfs onto my Ubuntu Machines[/URL] (This is optional to compiling YAFU, but sievers are necessary to run NFS routines.)


If you have already installed the following packages, you can skip the next two steps, although performing them wouldn't hurt anything.

Open a terminal and Enter:
[code]
sudo apt-get update
[/code]You should be prompted for your password. This prompt should only appear once for your terminal session, unless you leave it idle for a long time.

After the update completes and the user prompt reappears, type:
[code]
sudo apt-get install g++ m4 zlib1g-dev make p7zip subversion
[/code]Accept the prompt.

[B]Special Note on the following procedure(s):[/B] There are two branches available for YAFU. The "wip" branch and the "trunk" branch. The wip branch is the development branch and has all the latest work. This is the branch you should use if you want to stay at the leading edge of YAFU. However, if you have troubles with the installation, or just want a stable running package, you should use the trunk branch. I will note any differences in the installation steps with [B]wip/trunk[/B] designations.

To download the "[B]wip[/B]" YAFU package:
[code]
svn co https://svn.code.sf.net/p/yafu/code/branches/wip $HOME/Math/yafu
[/code]To download the "[B]trunk[/B]" YAFU package:
[code]
svn co https://svn.code.sf.net/p/yafu/code/trunk $HOME/Math/yafu
[/code]Move into the yafu folder:
[code]
cd Math/yafu
[/code]Using a text editor, open the Makefile file and make the following edits:
[code]
. . .
[B]#CC = gcc-7.3.0[/B] (This edit is only needed in the "wip" Makefile.)
. . .

# modify the following line for your particular msieve installation
. . .
else
[B]LIBS += -L../msieve[/B]
endif
. . .
[/code]Save and close Makefile.

Run the following in your terminal:
[code]
lscpu
[/code]See how many CPU(s) are reported and then look for sse4_1 and avx2 in the Flags listing.

Run the following in your terminal:
[code]
echo $HOME
[/code]Run the following in your terminal:
[code]
which ecm
[/code]You should see:
[code]
/usr/local/bin/ecm
[/code]Open the yafu.ini file and modify the following lines:
[code]
threads=<# of CPUs reported above>
ggnfs_dir=<$HOME string returned>/Math/ggnfs/bin/
ecm_path=/usr/local/bin/ecm
[/code]Save and close yafu.ini

For the "[B]wip[/B]" version, run the following in the terminal:
[code]
make NFS=1
[/code]or
[code]
make NFS=1 USE_SSE41=1
[/code]or
[code]
make NFS=1 USE_AVX2=1
[/code]or
[code]
make NFS=1 USE_SSE41=1 USE_AVX2=1
[/code]depending on the flags reported above.

For the "[B]trunk[/B]" version, run the following in the terminal:
[code]
make x86_64 NFS=1
[/code]or
[code]
make x86_64 NFS=1 USE_SSE41=1
[/code]or
[code]
make x86_64 NFS=1 USE_AVX2=1
[/code]or
[code]
make x86_64 NFS=1 USE_SSE41=1 USE_AVX2=1
[/code]depending on the flags reported above.

Test your installation and calculate tune data with the following command:
[code]
./yafu "tune"
[/code]Return to your home directory:
[code]
cd ~/
[/code]

bsquared 2018-02-24 02:01

[QUOTE=EdH;480734]
Run the following in the terminal:
[code]
make NFS=1
[/code]or
[code]
make NFS=1 USE_SSE41=1
[/code]depending on whether the flag was reported above.

[/QUOTE]

or
[code]
make NFS=1 USE_AVX2=1
[/code]depending on whether the flag was reported above.

This helps quite a bit for siqs on supported cpus.

EdH 2018-02-24 05:14

Excellent! Thank you B[SUP]2[/SUP].

Are sse4_1 and avx2 mutually exclusive?

uglyb 2018-04-14 21:49

What's about [B]ggnfs[/B]? Does yafu need it?

LaurV 2018-04-21 09:34

Yes, if you want to factor anything over 110 digits, you will need external ggnfs and path to it in yafu ini

MisterBitcoin 2018-04-28 18:12

Invalid option "B1pm1" while starting "tune".

Looks like somethink went wrong. After several hours following your posts.
At least GMP, ECM and msieve worked.



Any idea how I can fix that issue, so that yafu will work on my server?

EdH 2018-04-29 03:39

[QUOTE=MisterBitcoin;486491]Invalid option "B1pm1" while starting "tune".

Looks like somethink went wrong. After several hours following your posts.
At least GMP, ECM and msieve worked.



Any idea how I can fix that issue, so that yafu will work on my server?[/QUOTE]
Try the following:
[code]
$ which ecm
[/code]and make sure you get the line:
[code]
ecm: /usr/local/bin/ecm
[/code]This ensures it was installed to that location.

(If you get a different location, try that in your yafu.ini file in the next step.)

Next, make sure your yafu.ini file has the following line:
[code]
ecm_path=/usr/local/bin/ecm
[/code]On Debian I don't have sudo set, so I have to go into root mode via su to use make install with GMP and ECM, whereas in Ubuntu, I use sudo make install:

Debian:
[code]
$ make
$ make check
$ su
Password: **************
# make install
exit
$
[/code]Ubuntu:
[code]
$ make
$ make check
$ sudo make install
[sudo] password for user: **************
$
[/code]YAFU will do some ECM without the correct path in the ini file, but it

MisterBitcoin 2018-04-29 08:49

I´ve restarted the process but still got the same result.
ECM is in the correct path, same as ggnfs.



This what I see when I open yafu.ini.

[CODE]B1pm1=100000
B1pp1=20000
B1ecm=11000
rhomax=1000
threads=4
pretest_ratio=0.25
%ggnfs_dir=..\ggnfs-bin\Win32\
ggnfs_dir=/home/user/Math/ggnfs/bin/
%ggnfs_dir=../../ggnfs-bin/
%ecm_path=..\gmp-ecm\build.vc10\x64\Release\ecm.exe
ecm_path=/usr/local/bin/ecm[/CODE]


At least I´m happy that ggnfs went fine without problems. :smile:

bsquared 2018-04-29 15:56

[QUOTE=MisterBitcoin;486491]Invalid option "B1pm1" while starting "tune".

Looks like somethink went wrong. After several hours following your posts.
At least GMP, ECM and msieve worked.



Any idea how I can fix that issue, so that yafu will work on my server?[/QUOTE]

Is this yafu's tune that it's having problems with? That's something I haven't seen before, but, tune is optional. Just omit that step and see if you can run ecm curves with yafu.

./yafu "ecm(rsa(256),100)" -v -threads 2 -B1ecm 1000000

MisterBitcoin 2018-04-29 20:48

[QUOTE=bsquared;486579]Is this yafu's tune that it's having problems with? That's something I haven't seen before, but, tune is optional. Just omit that step and see if you can run ecm curves with yafu.

./yafu "ecm(rsa(256),100)" -v -threads 2 -B1ecm 1000000[/QUOTE]


Same result.

EdH 2018-04-30 03:17

The next thing I would try is to use the trunk version of YAFU:

First, rename your current yafu directory to yafu-orig:
[code]
mv ~/Math/yafu ~/Math/yafu-orig
[/code]Next, checkout the code/trunk versin of YAFU:
[code]
svn co https://svn.code.sf.net/p/yafu/code/trunk $HOME/Math/yafu
[/code]Next, move into the Math/yafu directory and edit the Makefile:

Change:
[code]
else
OBJ_EXT = .o

INC += -I../gmp/include
LIBS += -L../gmp/lib/linux/x86_64

INC += -I../gmp-ecm/include/linux
LIBS += -L../gmp-ecm/lib/linux/x86_64
[/code]To:
[code]
else
OBJ_EXT = .o

INC += -I[B]/usr/local/lib/[/B]
LIBS += -L[B]/usr/local/lib/[/B]

[B]#[/B]INC += -I../gmp-ecm/include/linux
[B]#[/B]LIBS += -L../gmp-ecm/lib/linux/x86_64
[/code]Next, change the msieve reference:
[code]
# modify the following line for your particular msieve installation

ifeq ($(MIC),1)
LIBS += -L../msieve/lib/phi
else
LIBS += -L../[B]msieve[/B]
endif
LIBS += -lmsieve
-L../msieve[/code]Now, save and close Makefile and copy your modified yafu.ini from yafu-orig into yafu, replacing the current one:
[code]
cp ~/Math/yafu-orig/yafu.ini ~/Math/yafu/yafu.ini
[/code]Check for sse4_1 and avx2:
[code]
cat /proc/cpuinfo
[/code]Next, compile yafu:
[code]
make x86_64 NFS=1 USE_SSE41=1 USE_AVX2=1
[/code]depending on the available functions.

And, finally test:
[code]
./yafu "tune"
[/code]

MisterBitcoin 2018-04-30 07:31

Once again failture.
Pure ECM works, so I believe there must be an linking error. Yafu searches for ECM, but it is unable to find it.

I put the correct path into yafu.ini, somehow it won´t work. You can find the ini earlier in this post.

If someone wants to take a look into the server, feel free to ask. I´ll give you VNC access.

EdH 2018-04-30 14:50

1 Attachment(s)
[QUOTE=MisterBitcoin;486636]Once again failture.
Pure ECM works, so I believe there must be an linking error. Yafu searches for ECM, but it is unable to find it.

I put the correct path into yafu.ini, somehow it won´t work. You can find the ini earlier in this post.

If someone wants to take a look into the server, feel free to ask. I´ll give you VNC access.[/QUOTE]
I might be able to take a look in a little while, if you'd like to PM me with the details.

Meanwhile, I've attached two instances of YAFU that were compiled on an i7-2600 running Debian. One is from the wip branch and the other from the trunk. Both run fine on other machines here, including a Fedora machine.

EdH 2018-05-02 14:06

@MisterBitcoin: I think it's running now. YAFU wouldn't even run a shell, due to something corrupted in the yafu.ini file. That's why it gave the error message dealing with B1pm1 failure, the first option.

VictordeHolland 2018-05-14 01:52

[QUOTE=EdH;486662]
Meanwhile, I've attached two instances of YAFU that were compiled on an i7-2600 running Debian. One is from the wip branch and the other from the trunk. Both run fine on other machines here, including a Fedora machine.[/QUOTE]
The trunk executable in your attachment works on Ubuntu 16.04.4 LTS!

I have most of my factoring programs-executables in <home>/factoring/ like:
/CADO-NFS/
/GGNFS/
/gmp/
/GMP-ECM/
/msieve/
/YAFU/

This is the yafu.ini I use: (I deleted the %windows dir/paths as they are confusing).
[code]B1pm1=100000
B1pp1=20000
B1ecm=11000
rhomax=1000
threads=32
pretest_ratio=0.25
ggnfs_dir=/home/victor/factoring/GGNFS/
ecm_path=/home/victor/factoring/GMP-ECM/ecm
[/code]Everything I checked works (rho, P-1, ECM, SIQS):
[code]05/14/18 02:17:14 v1.34.5 @ Supermicro, ****************************
05/14/18 02:17:14 v1.34.5 @ Supermicro, Starting factorization of 2056802480868100646375721251575555494408897387375737955882170045672576386016591560879707933101909539325829251496440620798637813
05/14/18 02:17:14 v1.34.5 @ Supermicro, using pretesting plan: normal
05/14/18 02:17:14 v1.34.5 @ Supermicro, no tune info: using qs/gnfs crossover of 95 digits
05/14/18 02:17:14 v1.34.5 @ Supermicro, ****************************
05/14/18 02:17:14 v1.34.5 @ Supermicro, rho: x^2 + 3, starting 1000 iterations on C127
05/14/18 02:17:14 v1.34.5 @ Supermicro, prp6 = 280673
05/14/18 02:17:14 v1.34.5 @ Supermicro, rho: x^2 + 3, starting 1000 iterations on C121
05/14/18 02:17:14 v1.34.5 @ Supermicro, rho: x^2 + 2, starting 1000 iterations on C121
05/14/18 02:17:14 v1.34.5 @ Supermicro, rho: x^2 + 1, starting 1000 iterations on C121
05/14/18 02:17:14 v1.34.5 @ Supermicro, pm1: starting B1 = 150K, B2 = gmp-ecm default on C121
05/14/18 02:17:14 v1.34.5 @ Supermicro, prp12 = 598990818061
05/14/18 02:17:14 v1.34.5 @ Supermicro, current ECM pretesting depth: 0.00
05/14/18 02:17:14 v1.34.5 @ Supermicro, scheduled 30 curves at B1=2000 toward target pretesting depth of 33.85
05/14/18 02:17:15 v1.34.5 @ Supermicro, prp13 = 4527716228491 (curve 1 stg2 B1=2000 sigma=451473622 thread=1)
05/14/18 02:17:15 v1.34.5 @ Supermicro, prp10 = 2756163353 (curve 1 stg2 B1=2000 sigma=2019115514 thread=3)
05/14/18 02:17:15 v1.34.5 @ Supermicro, Finished 32 curves using Lenstra ECM method on C110 input, B1=2K, B2=gmp-ecm default
05/14/18 02:17:15 v1.34.5 @ Supermicro, current ECM pretesting depth: 15.19
05/14/18 02:17:15 v1.34.5 @ Supermicro, scheduled 74 curves at B1=11000 toward target pretesting depth of 26.77
05/14/18 02:17:15 v1.34.5 @ Supermicro, prp18 = 248158049830971629 (curve 1 stg1 B1=11000 sigma=971411753 thread=27)
05/14/18 02:17:15 v1.34.5 @ Supermicro, Finished 32 curves using Lenstra ECM method on C87 input, B1=11K, B2=gmp-ecm default
05/14/18 02:17:15 v1.34.5 @ Supermicro, current ECM pretesting depth: 17.35
05/14/18 02:17:15 v1.34.5 @ Supermicro, scheduled 42 curves at B1=11000 toward target pretesting depth of 21.54
05/14/18 02:17:15 v1.34.5 @ Supermicro, Finished 64 curves using Lenstra ECM method on C70 input, B1=11K, B2=gmp-ecm default
05/14/18 02:17:15 v1.34.5 @ Supermicro, current ECM pretesting depth: 20.31
05/14/18 02:17:15 v1.34.5 @ Supermicro, scheduled 54 curves at B1=50000 toward target pretesting depth of 21.54
05/14/18 02:17:16 v1.34.5 @ Supermicro, Finished 64 curves using Lenstra ECM method on C70 input, B1=50K, B2=gmp-ecm default
05/14/18 02:17:16 v1.34.5 @ Supermicro, final ECM pretested depth: 21.81
05/14/18 02:17:16 v1.34.5 @ Supermicro, scheduler: switching to sieve method
05/14/18 02:17:16 v1.34.5 @ Supermicro, starting SIQS on c70: 3950565477929583959090443657115834369563721138106225828250209513731263
05/14/18 02:17:16 v1.34.5 @ Supermicro, random seeds: 2053521748, 760562015
05/14/18 02:17:16 v1.34.5 @ Supermicro, ==== sieve params ====
05/14/18 02:17:16 v1.34.5 @ Supermicro, n = 71 digits, 233 bits
05/14/18 02:17:16 v1.34.5 @ Supermicro, factor base: 12160 primes (max prime = 278671)
05/14/18 02:17:16 v1.34.5 @ Supermicro, single large prime cutoff: 22293680 (80 * pmax)
05/14/18 02:17:16 v1.34.5 @ Supermicro, double large prime range from 37 to 44 bits
05/14/18 02:17:16 v1.34.5 @ Supermicro, double large prime cutoff: 16854938548958
05/14/18 02:17:16 v1.34.5 @ Supermicro, allocating 5 large prime slices of factor base
05/14/18 02:17:16 v1.34.5 @ Supermicro, buckets hold 2048 elements
05/14/18 02:17:16 v1.34.5 @ Supermicro, using SSE4.1 enabled 32k sieve core
05/14/18 02:17:16 v1.34.5 @ Supermicro, sieve interval: 6 blocks of size 32768
05/14/18 02:17:16 v1.34.5 @ Supermicro, polynomial A has ~ 9 factors
05/14/18 02:17:16 v1.34.5 @ Supermicro, using multiplier of 3
05/14/18 02:17:16 v1.34.5 @ Supermicro, using SPV correction of 19 bits, starting at offset 30
05/14/18 02:17:16 v1.34.5 @ Supermicro, trial factoring cutoff at 78 bits
05/14/18 02:17:16 v1.34.5 @ Supermicro, ==== sieving started (32 threads) ====
05/14/18 02:17:18 v1.34.5 @ Supermicro, trial division touched 1832182 sieve locations out of 16130113536
05/14/18 02:17:18 v1.34.5 @ Supermicro, squfof: 1 failures, 17917 attempts, 321456 outside range, 154968 prp, 17172 useful
05/14/18 02:17:18 v1.34.5 @ Supermicro, 14496 relations found: 5464 full + 9032 from 81510 partial, using 41021 polys (322 A polys)
05/14/18 02:17:18 v1.34.5 @ Supermicro, on average, sieving found 2.12 rels/poly and 33699.23 rels/sec
05/14/18 02:17:18 v1.34.5 @ Supermicro, trial division touched 1832182 sieve locations out of 16130113536
05/14/18 02:17:18 v1.34.5 @ Supermicro, ==== post processing stage (msieve-1.38) ====
05/14/18 02:17:18 v1.34.5 @ Supermicro, begin with 86974 relations
05/14/18 02:17:18 v1.34.5 @ Supermicro, reduce to 24385 relations in 5 passes
05/14/18 02:17:18 v1.34.5 @ Supermicro, recovered 24385 relations
05/14/18 02:17:18 v1.34.5 @ Supermicro, recovered 18407 polynomials
05/14/18 02:17:18 v1.34.5 @ Supermicro, attempting to build 14496 cycles
05/14/18 02:17:18 v1.34.5 @ Supermicro, found 14496 cycles in 2 passes
05/14/18 02:17:18 v1.34.5 @ Supermicro, distribution of cycle lengths:
05/14/18 02:17:18 v1.34.5 @ Supermicro, length 1 : 5464
05/14/18 02:17:18 v1.34.5 @ Supermicro, length 2 : 6196
05/14/18 02:17:18 v1.34.5 @ Supermicro, length 3 : 2015
05/14/18 02:17:18 v1.34.5 @ Supermicro, length 4 : 610
05/14/18 02:17:18 v1.34.5 @ Supermicro, length 5 : 168
05/14/18 02:17:18 v1.34.5 @ Supermicro, length 6 : 31
05/14/18 02:17:18 v1.34.5 @ Supermicro, length 7 : 10
05/14/18 02:17:18 v1.34.5 @ Supermicro, length 9+: 2
05/14/18 02:17:18 v1.34.5 @ Supermicro, largest cycle: 8 relations
05/14/18 02:17:18 v1.34.5 @ Supermicro, matrix is 12160 x 14496 (2.3 MB) with weight 476375 (32.86/col)
05/14/18 02:17:18 v1.34.5 @ Supermicro, sparse part has weight 476375 (32.86/col)
05/14/18 02:17:18 v1.34.5 @ Supermicro, filtering completed in 4 passes
05/14/18 02:17:18 v1.34.5 @ Supermicro, matrix is 10983 x 11047 (1.5 MB) with weight 309420 (28.01/col)
05/14/18 02:17:18 v1.34.5 @ Supermicro, sparse part has weight 309420 (28.01/col)
05/14/18 02:17:18 v1.34.5 @ Supermicro, saving the first 48 matrix rows for later
05/14/18 02:17:18 v1.34.5 @ Supermicro, matrix is 10935 x 11047 (1.3 MB) with weight 248927 (22.53/col)
05/14/18 02:17:18 v1.34.5 @ Supermicro, sparse part has weight 220626 (19.97/col)
05/14/18 02:17:18 v1.34.5 @ Supermicro, matrix includes 64 packed rows
05/14/18 02:17:18 v1.34.5 @ Supermicro, commencing Lanczos iteration
05/14/18 02:17:18 v1.34.5 @ Supermicro, memory use: 1.7 MB
05/14/18 02:17:19 v1.34.5 @ Supermicro, lanczos halted after 174 iterations (dim = 10932)
05/14/18 02:17:19 v1.34.5 @ Supermicro, recovered 15 nontrivial dependencies
05/14/18 02:17:19 v1.34.5 @ Supermicro, prp36 = 117445937227520353139789517076610399
05/14/18 02:17:19 v1.34.5 @ Supermicro, prp35 = 33637310674071348724927955857253537
05/14/18 02:17:19 v1.34.5 @ Supermicro, Lanczos elapsed time = 1.0024 seconds.
05/14/18 02:17:19 v1.34.5 @ Supermicro, Sqrt elapsed time = 0.0142 seconds.
05/14/18 02:17:19 v1.34.5 @ Supermicro, SIQS elapsed time = 3.5977 seconds.[/code]NFS works too:
[code]05/14/18 02:23:52 v1.34.5 @ Supermicro, nfs: commencing nfs on c94: 9379745492489847473195765085744210645855556204246905462578925932774371960871599319713301154409
05/14/18 02:23:53 v1.34.5 @ Supermicro, nfs: commencing poly selection with 32 threads
05/14/18 02:23:53 v1.34.5 @ Supermicro, nfs: setting deadline of 20 seconds
05/14/18 02:24:21 v1.34.5 @ Supermicro, nfs: completed 523 ranges of size 250 in 28.4503 seconds
05/14/18 02:24:21 v1.34.5 @ Supermicro, nfs: best poly = # norm 6.654414e-13 alpha -5.486082 e 2.519e-08 rroots 2
05/14/18 02:24:21 v1.34.5 @ Supermicro, nfs: commencing lattice sieving with 32 threads
05/14/18 02:24:48 v1.34.5 @ Supermicro, nfs: commencing lattice sieving with 32 threads
05/14/18 02:25:14 v1.34.5 @ Supermicro, nfs: commencing lattice sieving with 32 threads
05/14/18 02:25:40 v1.34.5 @ Supermicro, nfs: commencing lattice sieving with 32 threads
05/14/18 02:26:03 v1.34.5 @ Supermicro, nfs: commencing lattice sieving with 32 threads
05/14/18 02:26:29 v1.34.5 @ Supermicro, nfs: commencing lattice sieving with 32 threads
05/14/18 02:26:55 v1.34.5 @ Supermicro, nfs: commencing msieve filtering
05/14/18 02:27:33 v1.34.5 @ Supermicro, nfs: raising min_rels by 5.00 percent to 2072441
05/14/18 02:27:33 v1.34.5 @ Supermicro, nfs: commencing lattice sieving with 32 threads
05/14/18 02:27:56 v1.34.5 @ Supermicro, nfs: commencing msieve filtering
05/14/18 02:28:42 v1.34.5 @ Supermicro, nfs: raising min_rels by 5.00 percent to 2410184
05/14/18 02:28:42 v1.34.5 @ Supermicro, nfs: commencing lattice sieving with 32 threads
05/14/18 02:29:07 v1.34.5 @ Supermicro, nfs: commencing msieve filtering
05/14/18 02:30:05 v1.34.5 @ Supermicro, nfs: commencing msieve linear algebra
05/14/18 02:33:02 v1.34.5 @ Supermicro, nfs: commencing msieve sqrt
05/14/18 02:33:54 v1.34.5 @ Supermicro, prp42 = 859735020008609871428759089831769060699941
05/14/18 02:33:54 v1.34.5 @ Supermicro, prp53 = 10910042366770069935194172108679294830357131379375349
05/14/18 02:33:54 v1.34.5 @ Supermicro, NFS elapsed time = 601.2586 seconds.[/code]

EdH 2018-05-14 02:58

[QUOTE=VictordeHolland;487524]The trunk executable in your attachment works on Ubuntu 16.04.4 LTS![/QUOTE]Thanks! I'm using the wip version everywhere to try to stay on the newest edge. Some of my machines are still using Ubuntu 14.04, but most are 16.04, with a handful of Debian systems. Of note, although I have not gathered empirical data to support this, the Debian systems appear to run a bit faster.

Very minor, but the wip version gives the proper tune info message when it runs:
[code]
fac: using tune info for qs/gnfs crossover
[/code]The last time I ran the trunk version, it wouldn't give the proper message, even when the tune info was available.
[code]
fac: no tune info: using qs/gnfs crossover of 95 digits
[/code]I still see this message with one of my machines running the wip version, too. I see you don't have tune info yet, but when you do, I'd be interested in what message is reported during the run?

[QUOTE=VictordeHolland;487524]I have most of my factoring programs-executables in <home>/factoring/ like:
/CADO-NFS/
/GGNFS/
/gmp/
/GMP-ECM/
/msieve/
/YAFU/[/QUOTE]I see my difference is Math where you have factoring and I use all lower case for my package directories. I've also shortened my GMP-ECM to ecm, since that is the name used to run it.

[QUOTE=VictordeHolland;487524]This is the yafu.ini I use: (I deleted the %windows dir/paths as they are confusing).
[code]B1pm1=100000
B1pp1=20000
B1ecm=11000
rhomax=1000
threads=32
pretest_ratio=0.25
ggnfs_dir=/home/victor/factoring/GGNFS/
ecm_path=/home/victor/factoring/GMP-ECM/ecm
[/code][/QUOTE]I also delete all the %entries, but they are ignored, if left. MisterBitcoin had a problem with his .ini, that I suspect was caused by a Windows editor adding an incompatible character somewhere. I even tried cleaning up the one he had, but the problem didn't clear up until I started fresh with a linux editor. I don't remember for sure, but I think I used vi.

I do see a couple differences in your .ini that I have changed in mine. First, I have the gnfs-* binaries in ggnfs/bin/. Second, if you have installed GMP-ECM, the executable should reside in /usr/local/bin/. This can be proven with:
[code]
$ whereis ecm
ecm: /usr/local/bin/ecm
[/code]So, in my .ini file I have the line:
[code]
ecm_path=/usr/local/bin/ecm
[/code]Thanks for your post!

vochong 2019-09-03 05:55

Please help fix the yafu compile errors
 
Hi Ed,

I followed your instructions to install ecm, ggns, gmps, msieve successfully. However, when it came to compiling yafu, it kept failing with the following error. Could you please give me some suggestion to fix it.

Best regards,
Eric

Ubuntu 18.04.3

[CODE]~/Math$ ls -la

drwxrwxr-x 27 python python 12288 Sep 2 03:24 ecm
drwxrwxr-x 11 python python 4096 Apr 23 2011 ggnfs
drwxr-xr-x 16 python python 4096 Sep 2 10:54 gmp
drwxrwxr-x 21 python python 4096 Sep 2 22:45 msieve
drwxrwxr-x 11 python python 4096 Sep 2 22:41 yafu

cd yafu/

~/Math/yafu$ make NFS=1 USE_SSE41=1

gcc -g -DUSE_SSE41 -m64 -msse4.1 -DUSE_NFS -O3 -fomit-frame-pointer -Wall -I. -Iinclude -Itop/aprcl -Itop/ -I../msieve/zlib -I/usr/local/lib/ -c -o factor/qs/msieve/lanczos.o factor/qs/msieve/lanczos.c
In file included from factor/qs/msieve/lanczos.c:20:0:
include/qs.h:30:10: fatal error: cofactorize.h: No such file or directory
#include "cofactorize.h"
^~~~~~~~~~~~~~~
compilation terminated.
Makefile:370: recipe for target 'factor/qs/msieve/lanczos.o' failed
make: *** [factor/qs/msieve/lanczos.o] Error 1[/CODE]

bsquared 2019-09-03 14:33

[QUOTE=vochong;525062]Hi Ed,

I followed your instructions to install ecm, ggns, gmps, msieve successfully. However, when it came to compiling yafu, it kept failing with the following error. Could you please give me some suggestion to fix it.

Best regards,
Eric

Ubuntu 18.04.3

[CODE]~/Math$ ls -la

drwxrwxr-x 27 python python 12288 Sep 2 03:24 ecm
drwxrwxr-x 11 python python 4096 Apr 23 2011 ggnfs
drwxr-xr-x 16 python python 4096 Sep 2 10:54 gmp
drwxrwxr-x 21 python python 4096 Sep 2 22:45 msieve
drwxrwxr-x 11 python python 4096 Sep 2 22:41 yafu

cd yafu/

~/Math/yafu$ make NFS=1 USE_SSE41=1

gcc -g -DUSE_SSE41 -m64 -msse4.1 -DUSE_NFS -O3 -fomit-frame-pointer -Wall -I. -Iinclude -Itop/aprcl -Itop/ -I../msieve/zlib -I/usr/local/lib/ -c -o factor/qs/msieve/lanczos.o factor/qs/msieve/lanczos.c
In file included from factor/qs/msieve/lanczos.c:20:0:
include/qs.h:30:10: fatal error: cofactorize.h: No such file or directory
#include "cofactorize.h"
^~~~~~~~~~~~~~~
compilation terminated.
Makefile:370: recipe for target 'factor/qs/msieve/lanczos.o' failed
make: *** [factor/qs/msieve/lanczos.o] Error 1[/CODE][/QUOTE]

Unfortunately this probably only works with AVX2 if you are using the wip version. So, you'd need a computer new enough to have AVX2 instructions (haswell or newer) and then build with USE_AVX2=1.

EdH 2019-09-03 14:52

Thanks for helping out bsquared!


Hi Eric,

This is the trouble with trying to stay on the edge. Sometimes it changes slightly and you slip off.

Here's what you should do, and I'll probably change the post to this:

Instead of using the /wip version which has the latest and greatest, use the following command to acquire the main release (/trunk) version of YAFU:

First, I'd get rid of the current yafu folder. Then use the following in place of the /wip checkout:
[code]
svn co https://svn.code.sf.net/p/yafu/code/trunk $HOME/Math/yafu
[/code]The rest should be pretty much the same, but let me know if it doesn't work. I do see I forgot to mention ggnfs in this thread, though.

Thanks for helping me stay on top of what works (and doesn't).


-Ed

vochong 2019-09-04 09:30

Ed's modified instructions work perfectly.
 
Thank you Ed and bsquared!.

I followed Ed's newly modified instructions and was able to compile Yafu the very first time.

Last year, I tried to install Yafu on another machine using Ed's original instructions and everything worked smoothly, so it was disconcerting to me why I encountered this error on this machine.

You help bring the joy of mathematics and computing to the mass. Thank you so much!

Best regards,
Eric

[QUOTE=EdH;525080]Thanks for helping out bsquared!


Hi Eric,

This is the trouble with trying to stay on the edge. Sometimes it changes slightly and you slip off.

Here's what you should do, and I'll probably change the post to this:

Instead of using the /wip version which has the latest and greatest, use the following command to acquire the main release (/trunk) version of YAFU:

First, I'd get rid of the current yafu folder. Then use the following in place of the /wip checkout:
[code]
svn co https://svn.code.sf.net/p/yafu/code/trunk $HOME/Math/yafu
[/code]The rest should be pretty much the same, but let me know if it doesn't work. I do see I forgot to mention ggnfs in this thread, though.

Thanks for helping me stay on top of what works (and doesn't).


-Ed[/QUOTE]

EdH 2019-09-04 15:44

Glad to hear it's working now, Eric. Thank you for letting me know it wasn't. Although I test my instructions whenever I set up a "new" machine, I hadn't done so in a little while and I should, since I know bsquared is always staying on top of current capabilities with the /wip branch of YAFU. -Ed

dizzyneuron 2019-09-04 21:53

Hello

Been trying to build YAFU but seem to be stuck. Think maybe something with makefile but old brain is rusty so hoping one can point me in right direction -

[CODE]../msieve/libmsieve.a(savefile.o): In function `savefile_open':
savefile.c:(.text+0x12b): undefined reference to `gzopen64'
savefile.c:(.text+0x24b): undefined reference to `gzopen64'
savefile.c:(.text+0x291): undefined reference to `gzopen64'
../msieve/libmsieve.a(savefile.o): In function `savefile_close':
savefile.c:(.text+0x36e): undefined reference to `gzclose'
../msieve/libmsieve.a(savefile.o): In function `savefile_write_line':
savefile.c:(.text+0x478): undefined reference to `gzputs'
../msieve/libmsieve.a(savefile.o): In function `savefile_flush':
savefile.c:(.text+0x4c5): undefined reference to `gzputs'
../msieve/libmsieve.a(savefile.o): In function `savefile_eof':
savefile.c:(.text+0x39d): undefined reference to `gzeof'
../msieve/libmsieve.a(savefile.o): In function `savefile_read_line':
savefile.c:(.text+0x41c): undefined reference to `gzgets'
../msieve/libmsieve.a(savefile.o): In function `savefile_rewind':
savefile.c:(.text+0x51d): undefined reference to `gzrewind'
collect2: error: ld returned 1 exit status
Makefile:293: recipe for target 'x86_64' failed
make: *** [x86_64] Error 1[/CODE]

Running Ubuntu 18.04 and all other installs built well. Thanks.

Cheers, Michael

EdH 2019-09-05 14:27

Hi Michael,

It looks like you didn't include NO_ZLIB=1 in your msieve compilation. YAFU doesn't recognize the ZLIB calls.

Try going back to your msieve directory and issue the following:
[code]
make clean
make all ECM=1 NO_ZLIB=1 <any other switches>
[/code]Ed

dizzyneuron 2019-09-05 16:26

Hello Ed

Thank you, sorry I missed that instruction but all went well. Only issue seems to be with ./yafu "tune" it gives me a segmentation fault. But so far does not seem to cause problems. Thanks again for help your instructions helped alot to get all installed. Take care.

Cheers, Michael

EdH 2019-09-06 03:38

Hi Michael,


A segmentation fault means something crashed. You probably need to track that down. bsquared's help may be needed. Can you tell what operation happened when the fault occurred? SIQS/NFS?


Ed

dizzyneuron 2019-09-06 19:31

Hello Ed

This what I get -

Setting THREADS = 1 for tuning

starting SIQS on c60: 349594255864176572614071853194924838158088864370890996447417

==== sieving in progress (1 thread): 3568 relations needed ====
==== Press ctrl-c to abort and save state ====
1028 rels found: 702 full + 326 from 7135 partial, (9630.79 rels/sec)
Max specified relations found

sieve time = 0.0000, relation time = 0.0000, poly_time = 0.0000
trial division touched 145298 sieve locations out of 1618477056


elapsed time for ~10k relations of c60 = 1.1925 seconds.
extrapolated time for complete factorization = 1.7780 seconds

starting SIQS on c65: 34053408309992030649212497354061832056920539397279047809781589871

==== sieving in progress (1 thread): 6448 relations needed ====
==== Press ctrl-c to abort and save state ====
Segmentation fault

Thank you.

Cheers, Michael

EdH 2019-09-06 23:10

Hi Michael,

It looks like SIQS is what crashed. I'm going to have to defer any thoughts to bsquared. Hopefully he'll catch the posts.

I guess the only other thing I might try would be recompiling msieve and YAFU.

Sorry I can't be of more help.

Ed

dizzyneuron 2019-09-07 20:24

Hello Ed

You have been a great help, I will try to recompile and hope. Take care.

Cheers, Michael

EdH 2019-09-07 21:51

[QUOTE=dizzyneuron;525409]Hello Ed

You have been a great help, I will try to recompile and hope. Take care.

Cheers, Michael[/QUOTE]
Hi Michael,

Keep me posted - good luck with it.

Ed

bsquared 2019-09-09 00:36

[QUOTE=dizzyneuron;525353]Hello Ed

This what I get -

Setting THREADS = 1 for tuning

starting SIQS on c60: 349594255864176572614071853194924838158088864370890996447417

==== sieving in progress (1 thread): 3568 relations needed ====
==== Press ctrl-c to abort and save state ====
1028 rels found: 702 full + 326 from 7135 partial, (9630.79 rels/sec)
Max specified relations found

sieve time = 0.0000, relation time = 0.0000, poly_time = 0.0000
trial division touched 145298 sieve locations out of 1618477056


elapsed time for ~10k relations of c60 = 1.1925 seconds.
extrapolated time for complete factorization = 1.7780 seconds

starting SIQS on c65: 34053408309992030649212497354061832056920539397279047809781589871

==== sieving in progress (1 thread): 6448 relations needed ====
==== Press ctrl-c to abort and save state ====
Segmentation fault

Thank you.

Cheers, Michael[/QUOTE]

Does it also crash if you try to factor this normally or just during tune?

dizzyneuron 2019-09-09 12:40

Hello

Seems to fail with factor to -

09/09/19 08:36:57 v1.34.5 @ dizzyneuron, System/Build Info:
Using GMP-ECM 7.0.4, Powered by GMP 6.1.2
detected Intel(R) Core(TM) i5-8300H CPU @ 2.30GHz
detected L1 = 32768 bytes, L2 = 8388608 bytes, CL = 64 bytes
measured cpu frequency ~= 2303.993260
using 1 random witnesses for Rabin-Miller PRP checks

===============================================================
======= Welcome to YAFU (Yet Another Factoring Utility) =======
======= bbuhrow at gmail.com =======
======= Type help at any time, or quit to quit =======
===============================================================
cached 78498 primes. pmax = 999983


>> factor(34053408309992030649212497354061832056920539397279047809781589871)

fac: factoring 34053408309992030649212497354061832056920539397279047809781589871
fac: using pretesting plan: normal
fac: no tune info: using qs/gnfs crossover of 95 digits

starting SIQS on c65: 34053408309992030649212497354061832056920539397279047809781589871

==== sieving in progress ( 8 threads): 6448 relations needed ====
==== Press ctrl-c to abort and save state ====
Segmentation fault

Hope this helps. Thank youfor your help.

Cheers, Michael

dizzyneuron 2019-09-10 17:55

Hello Ed/Ben

Looks like all is working now. Reread all post and did a make clean and then just used make x86_64 NFS=1, other flags show in cpu_info but am guessing they cause issues, cannot be sure may experimet more later. Thank both of you again for your help, funny brain still has issues with everyday things but math, with alot of stickies still makes my day. Take care.

Cheers, Michael

Ferrier 2019-10-24 16:04

[QUOTE=dizzyneuron;525571]Hello

Seems to fail with factor to -

09/09/19 08:36:57 v1.34.5 @ dizzyneuron, System/Build Info:
Using GMP-ECM 7.0.4, Powered by GMP 6.1.2
detected Intel(R) Core(TM) i5-8300H CPU @ 2.30GHz
detected L1 = 32768 bytes, L2 = 8388608 bytes, CL = 64 bytes
measured cpu frequency ~= 2303.993260
using 1 random witnesses for Rabin-Miller PRP checks

===============================================================
======= Welcome to YAFU (Yet Another Factoring Utility) =======
======= bbuhrow at gmail.com =======
======= Type help at any time, or quit to quit =======
===============================================================
cached 78498 primes. pmax = 999983


>> factor(34053408309992030649212497354061832056920539397279047809781589871)

fac: factoring 34053408309992030649212497354061832056920539397279047809781589871
fac: using pretesting plan: normal
fac: no tune info: using qs/gnfs crossover of 95 digits

starting SIQS on c65: 34053408309992030649212497354061832056920539397279047809781589871

==== sieving in progress ( 8 threads): 6448 relations needed ====
==== Press ctrl-c to abort and save state ====
Segmentation fault

Hope this helps. Thank youfor your help.

Cheers, Michael[/QUOTE]

Had almost the same issue after compiling the [B]trunk[/B] version.
Both in `./yafu "tune"` and normal factoring like `./yafu <a 70digits number>`


Though the EdH's compilation seems fine at #13...

Ferrier 2019-10-24 16:07

make option
 
[QUOTE=EdH;486662]I might be able to take a look in a little while, if you'd like to PM me with the details.

Meanwhile, I've attached two instances of YAFU that were compiled on an i7-2600 running Debian. One is from the wip branch and the other from the trunk. Both run fine on other machines here, including a Fedora machine.[/QUOTE]

I guess this binary is compiled with only NFS=1 option?

EdH 2019-10-25 00:20

[QUOTE=Ferrier;528797]I guess this binary is compiled with only NFS=1 option?[/QUOTE]
That is correct.

I think B[SUP]2[/SUP] will have to address the segmentation fault issue. I think I've seen it with the wip branch in the past, but not with the trunk branch.

Ferrier 2019-10-25 03:35

[QUOTE=dizzyneuron;525648]Hello Ed/Ben

Looks like all is working now. Reread all post and did a make clean and then just used make x86_64 NFS=1, other flags show in cpu_info but am guessing they cause issues, cannot be sure may experimet more later. Thank both of you again for your help, funny brain still has issues with everyday things but math, with alot of stickies still makes my day. Take care.

Cheers, Michael[/QUOTE]

Thanks for the info.

I managed to use the following command to recompile the trunk version and run [B]./yafu "tune"[/B] successfully (without [B]USE_AVX2=1[/B])
[CODE]
make clean
make x86_64 NFS=1 USE_SSE41=1
[/CODE]
on Intel Xeon Gold 6136(support sse4_1 and avx2)

EdH 2019-10-25 14:26

[QUOTE=Ferrier;528871]Thanks for the info.

I managed to use the following command to recompile the trunk version and run [B]./yafu "tune"[/B] successfully (without [B]USE_AVX2=1[/B])
[CODE]
make clean
make x86_64 NFS=1 USE_SSE41=1
[/CODE]on Intel Xeon Gold 6136(support sse4_1 and avx2)[/QUOTE]Glad you have something working. Bummer on the avx2. I think that is supposed to add considerably.

Thanks for the report. Hopefully B[SUP]2[/SUP] sees this and has a chance to chime in about the avx2 issue.

Ferrier 2019-10-25 16:53

Hi EdH,

I followed the instructions and get yafu working. But I noticed in [B]factor.log[/B] that when doing siqs factoring, it is still using msieve-1.38 which I think the latest version per your [URL="https://www.mersenneforum.org/showthread.php?t=23085"]msieve install guide[/URL] is 1.54

(I double checked that I have modified the msieve path in Makefile per your guide)


Any idea on this?

Thanks

EdH 2019-10-25 18:03

[QUOTE=Ferrier;528925]Hi EdH,

I followed the instructions and get yafu working. But I noticed in [B]factor.log[/B] that when doing siqs factoring, it is still using msieve-1.38 which I think the latest version per your [URL="https://www.mersenneforum.org/showthread.php?t=23085"]msieve install guide[/URL] is 1.54

(I double checked that I have modified the msieve path in Makefile per your guide)


Any idea on this?

Thanks[/QUOTE]
Hi Ferrier,

That text is hard coded into the factor/qs/SIQS.c source code. I don't think it represents the current version of msieve in any way. This is another question for B[SUP]2[/SUP].

-Ed

bsquared 2019-10-25 18:41

The Xeon Gold 6136 supports AVX512, but you will have to get the code in branches/wip to use it. To do that go here: [url]https://sourceforge.net/p/yafu/code/HEAD/tree/branches/wip/[/url] and click on download snapshot.

Should compile in linux with NFS=1 USE_AVX2=1 SKYLAKEX=1 assuming you have a recent version of gcc (I think 7.3 or better).

The msieve-1.38 that you see, as EdH mentioned, is hardcoded into the siqs codebase. It is only used to solve siqs matrices and is perfectly suited for that (newer versions of msieve wouldn't help). More recent versions of msieve are used in NFS jobs. So you are doing everything correct.

Ferrier 2019-10-29 01:42

wip compile error
 
[QUOTE=bsquared;528934]The Xeon Gold 6136 supports AVX512, but you will have to get the code in branches/wip to use it. To do that go here: [url]https://sourceforge.net/p/yafu/code/HEAD/tree/branches/wip/[/url] and click on download snapshot.

Should compile in linux with NFS=1 USE_AVX2=1 SKYLAKEX=1 assuming you have a recent version of gcc (I think 7.3 or better).

The msieve-1.38 that you see, as EdH mentioned, is hardcoded into the siqs codebase. It is only used to solve siqs matrices and is perfectly suited for that (newer versions of msieve wouldn't help). More recent versions of msieve are used in NFS jobs. So you are doing everything correct.[/QUOTE]

Hi B^2

I tried to compile yafu wip
[CODE]svn co https://svn.code.sf.net/p/yafu/code/branches/wip $HOME/Math/yafuwip[/CODE]

svn log shows the latest commit is [B]r379 2019-10-16 16:03:54[/B]


I modified the Makefile with msieve path and yafu.ini suggested by OP

trying to compile, had the same issue:
[url]https://www.mersenneforum.org/showpost.php?p=525079&postcount=18[/url]

But [B]lscpu[/B] shows Intel(R) Xeon(R) Gold 6136 CPU supporting avx2 avx512f avx512cd


Here is my output:
[CODE]
root@c3:~/Math/yafuwip# make NFS=1 USE_AVX2=1 SKYLAKEX=1
gcc -g -DUSE_AVX2 -DUSE_AVX512F -DUSE_AVX512BW -march=skylake-avx512 -DUSE_AVX2 -DUSE_SSE41 -mavx2 -DUSE_SSE41 -m64 -msse4.1 -DUSE_NFS -O3 -fomit-frame-pointer -Wall -I. -Iinclude -Itop/aprcl -Itop/ -I../msieve/zlib -I../gmp/include -I../gmp-ecm/include/ -c -o factor/qs/msieve/lanczos.o factor/qs/msieve/lanczos.c
In file included from factor/qs/msieve/lanczos.c:20:0:
include/qs.h:30:10: fatal error: cofactorize.h: No such file or directory
#include "cofactorize.h"
^~~~~~~~~~~~~~~
compilation terminated.
Makefile:369: recipe for target 'factor/qs/msieve/lanczos.o' failed
make: *** [factor/qs/msieve/lanczos.o] Error 1
[/CODE]

EdH 2019-10-29 02:20

Hi Ferrier,

This is actually a misplaced (or unregistered file - I'm not sure which) The fix is to move (or copy) cofactorize.h from the yafu/factor/qs** directory into the yafu/include directory and give that a try.

**I'm pretty sure that's where it was. If not, try to "locate" it.

-Ed

Ferrier 2019-10-29 03:00

[QUOTE=EdH;529156]Hi Ferrier,

This is actually a misplaced (or unregistered file - I'm not sure which) The fix is to move (or copy) cofactorize.h from the yafu/factor/qs** directory into the yafu/include directory and give that a try.

**I'm pretty sure that's where it was. If not, try to "locate" it.

-Ed[/QUOTE]

Thanks, EdH!

Ferrier 2019-10-29 04:28

[QUOTE=bsquared;528934]The Xeon Gold 6136 supports AVX512, but you will have to get the code in branches/wip to use it. To do that go here: [url]https://sourceforge.net/p/yafu/code/HEAD/tree/branches/wip/[/url] and click on download snapshot.

Should compile in linux with NFS=1 USE_AVX2=1 SKYLAKEX=1 assuming you have a recent version of gcc (I think 7.3 or better).

The msieve-1.38 that you see, as EdH mentioned, is hardcoded into the siqs codebase. It is only used to solve siqs matrices and is perfectly suited for that (newer versions of msieve wouldn't help). More recent versions of msieve are used in NFS jobs. So you are doing everything correct.[/QUOTE]

FWIW, after I compiled with [CODE]make NFS=1 USE_AVX2=1 SKYLAKEX=1[/CODE] in wip(r379).

Some observations:

Run ./yafu RSA-100 caused
[CODE]
root@c3c:~/Math/yafuwip# ./yafu 1522605027922533360535618378132637429718068114961380688657908494580122963258952897654000350692006139


fac: factoring 1522605027922533360535618378132637429718068114961380688657908494580122963258952897654000350692006139
fac: using pretesting plan: normal
fac: using specified qs/gnfs crossover of 93 digits
fac: using specified qs/snfs crossover of 75 digits
div: primes less than 10000
fmt: 1000000 iterations
rho: x^2 + 3, starting 200 iterations on C100 rho: x^2 + 2, starting 200 iterations on C100 rho: x^2 + 1, starting 200 iterations on C100 nfs: searching for brent special forms...
nfs: searching for homogeneous cunningham special forms...
nfs: searching for XYYXF special forms...
nfs: couldn't find special form
pm1: starting B1 = 150K, B2 = gmp-ecm default on C100
ecm: 30/30 curves on C100, B1=2k, B2=gmp-ecm default
ecm: 74/74 curves on C100, B1=11k, B2=gmp-ecm default
ecm: 214/214 curves on C100, B1=50k, B2=gmp-ecm default, ETA: 0 sec
pm1: starting B1 = 3750K, B2 = gmp-ecm default on C100
double free or corruption (out)
Aborted (core dumped)
[/CODE]

But run ./yafu 'factor(RSA-100)' had no issue.

./yafu 'siqs(some c95)' works too (with obvious [URL="https://www.mersenneforum.org/showpost.php?p=529163&postcount=29"]10% time saving[/URL] compared to trunk version!)

EdH 2019-10-29 13:44

I'm always using quotes around the basic operation, so I haven't noticed this trouble. Any options have to be outside the quotes, so maybe the quotes are a way to make sure they are parsed correctly. An example is how I get an SNFS polynomial:
[code]
./yafu "snfs(<Homogenous Cunningham number>,<cofactor to work on>)" -v -np
[/code]

bsquared 2019-10-29 14:02

[QUOTE=EdH;529189]I'm always using quotes around the basic operation, so I haven't noticed this trouble. Any options have to be outside the quotes, so maybe the quotes are a way to make sure they are parsed correctly. An example is how I get an SNFS polynomial:
[code]
./yafu "snfs(<Homogenous Cunningham number>,<cofactor to work on>)" -v -np
[/code][/QUOTE]

No, yafu is supposed to be able to handle input of just a number (the default "factor" function is applied in such cases). But there was a bug which I've just fixed. Thanks Ferrier for the report!

bsquared 2019-10-29 14:09

your snfs example is exactly right.

RedGolpe 2020-03-22 21:15

Hi all, I am trying to install yafu (trunk) on an Ubuntu subsystem but when I run[CODE]./yafu "tune"[/CODE]or any SIQS factorization I too obtain a segmentation fault error. Although cpuinfo reports both flags, I tried to recompile with[CODE]make clean
make x86_64 NFS=1[/CODE]
with no luck. I even tried [CODE]make clean
make x86_64[/CODE]The error happens during SIQS phase:[CODE]starting SIQS on c65: 34053408309992030649212497354061832056920539397279047809781589871

==== sieving in progress (12 threads): 6448 relations needed ====
==== Press ctrl-c to abort and save state ====
Segmentation fault (core dumped)[/CODE]Any suggestions? Thank you in advance.

EdH 2020-03-23 02:51

This is something B[SUP]2[/SUP] will have to answer, although you might try a little bit earlier revision and see if it works, until B[SUP]2[/SUP] can give some help.

If he doesn't happen to catch the issue in this thread, you might try posting a note in the [URL="https://www.mersenneforum.org/showthread.php?t=16667"]YAFU Bugs[/URL] thread

bsquared 2020-03-23 13:45

Hmm, I am not having any problems with this one.

Can you re-run with -v -v on the command line?

Also can you post your gcc version (gcc -v)?

RedGolpe 2020-03-23 14:10

Thank you both for the quick replies.

I have recompiled it with[CODE]make clean
make x86_64 NFS=1[/CODE]to stay on the safest side.

Here's the report of the -v -v run on the same C80 failing the "tune":[CODE]redgolpe@c2018:~/yafu$ ./yafu -v -v


03/23/20 15:06:31 v1.34.5 @ c2018, System/Build Info:
Using GMP-ECM 7.0.5-dev, Powered by GMP 6.2.0
detected Intel(R) Core(TM) i7-7800X CPU @ 3.50GHz
detected L1 = 32768 bytes, L2 = 8650752 bytes, CL = 64 bytes
measured cpu frequency ~= 3503.983800
using 1 random witnesses for Rabin-Miller PRP checks

===============================================================
======= Welcome to YAFU (Yet Another Factoring Utility) =======
======= bbuhrow@gmail.com =======
======= Type help at any time, or quit to quit =======
===============================================================
cached 78498 primes. pmax = 999983


>> factor(33727095233132290409342213138708322681737322487170896778164145844669592994743377)

fac: factoring 33727095233132290409342213138708322681737322487170896778164145844669592994743377
fac: using pretesting plan: normal
fac: no tune info: using qs/gnfs crossover of 95 digits
input from file = 33727095233132290409342213138708322681737322487170896778164145844669592994743377
input to yafu = 33727095233132290409342213138708322681737322487170896778164145844669592994743377
input matches with multiple of 1
fac: found siqs savefile, resuming siqs
input from file = 33727095233132290409342213138708322681737322487170896778164145844669592994743377
input to yafu = 33727095233132290409342213138708322681737322487170896778164145844669592994743377
input matches with multiple of 1

starting SIQS on c80: 33727095233132290409342213138708322681737322487170896778164145844669592994743377
fb bounds100%
small: 1024
SPV: 35
10bit: 96
11bit: 168
12bit: 304
13bit: 536
32k div 3: 688
14bit: 960
15bit: 1752
med: 2512
large: 14096
all: 47184
start primes
SPV: 241
10bit: 1033
11bit: 2089
12bit: 4261
13bit: 8317
32k div 3: 10987
14bit: 16451
15bit: 32789
med: 49307
large: 327571
restarting siqs from saved data set
0 relations found: 0 full + 0 from 0 partial
threw away 0 relations with large primes too small

==== sieve params ====
n = 80 digits, 265 bits
factor base: 47184 primes (max prime = 1218601)
single large prime cutoff: 109674090 (90 * pmax)
double large prime range from 41 to 49 bits
double large prime range from 1484988397201 to 296611012458251
allocating 7 large prime slices of factor base
buckets hold 2048 elements
using SSE2 enabled 32k sieve core
sieve interval: 10 blocks of size 32768
polynomial A has ~ 10 factors
using multiplier of 1
using SPV correction of 21 bits, starting at offset 35
trial factoring cutoff at 87 bits

==== sieving in progress (12 threads): 47248 relations needed ====
==== Press ctrl-c to abort and save state ====
Segmentation fault (core dumped)[/CODE]Output of gcc -v:[CODE]Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/7/lto-wrapper
OFFLOAD_TARGET_NAMES=nvptx-none
OFFLOAD_TARGET_DEFAULT=1
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu 7.5.0-3ubuntu1~18.04' --with-bugurl=file:///usr/share/doc/gcc-7/README.Bugs --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++ --prefix=/usr --with-gcc-major-version-only --program-suffix=-7 --program-prefix=x86_64-linux-gnu- --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --enable-bootstrap --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-vtable-verify --enable-libmpx --enable-plugin --enable-default-pie --with-system-zlib --with-target-system-zlib --enable-objc-gc=auto --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-offload-targets=nvptx-none --without-cuda-driver --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 7.5.0 (Ubuntu 7.5.0-3ubuntu1~18.04)[/CODE]

EdH 2020-03-23 14:57

I noticed that you had a siqs savefile that was grabbed by the current run. Not sure if it could cause troubles, but try removing all files from earlier runs (factor.log, session.log, siqs.dat). Also try running it without yafu.ini. Maybe, make a temp directory and copy only the yafu file into it and try it there.

I tried a fresh, current install on an 18.04 machine, copied yafu into a clean directory, and it ran fine:
[code]
$ ./yafu "factor(33727095233132290409342213138708322681737322487170896778164145844669592994743377)"
. . .
SIQS elapsed time = 113.6559 seconds.
Total factoring time = 135.7024 seconds


***factors found***

P40 = 3104758941649954796179093599696620654807
P41 = 10863031838216972467057273004441996191511
1
[/code]

RedGolpe 2020-03-23 15:13

That was just the leftover from the previous tune, which still failed. I gave it a shot anyway, but no luck:[CODE]redgolpe@c2018:~$ mkdir yafu-tmp
redgolpe@c2018:~$ cd yafu-tmp
redgolpe@c2018:~/yafu-tmp$ cp ../yafu/yafu .
redgolpe@c2018:~/yafu-tmp$ ls
yafu
redgolpe@c2018:~/yafu-tmp$ ./yafu -v -v


03/23/20 16:10:13 v1.34.5 @ c2018, System/Build Info:
Using GMP-ECM 7.0.5-dev, Powered by GMP 6.2.0
detected Intel(R) Core(TM) i7-7800X CPU @ 3.50GHz
detected L1 = 32768 bytes, L2 = 8650752 bytes, CL = 64 bytes
measured cpu frequency ~= 3503.987970
using 1 random witnesses for Rabin-Miller PRP checks

===============================================================
======= Welcome to YAFU (Yet Another Factoring Utility) =======
======= bbuhrow@gmail.com =======
======= Type help at any time, or quit to quit =======
===============================================================
cached 78498 primes. pmax = 999983


>> factor(33727095233132290409342213138708322681737322487170896778164145844669592994743377)

fac: factoring 33727095233132290409342213138708322681737322487170896778164145844669592994743377
fac: using pretesting plan: normal
fac: no tune info: using qs/gnfs crossover of 95 digits
div: primes less than 10000
rho: x^2 + 3, starting 1000 iterations on C80
rho: x^2 + 2, starting 1000 iterations on C80
rho: x^2 + 1, starting 1000 iterations on C80
pm1: starting B1 = 150K, B2 = gmp-ecm default on C80
fac: setting target pretesting digits to 24.62
fac: estimated sum of completed work is t0.00
fac: work done at B1=2000: 0 curves, max work = 30 curves
fac: 30 more curves at B1=2000 needed to get to t24.62
ecm: 30/30 curves on C80, B1=2K, B2=gmp-ecm default
fac: setting target pretesting digits to 24.62
fac: t15: 1.00
fac: t20: 0.04
fac: estimated sum of completed work is t15.18
fac: work done at B1=11000: 0 curves, max work = 74 curves
fac: 74 more curves at B1=11000 needed to get to t24.62
ecm: 74/74 curves on C80, B1=11K, B2=gmp-ecm default
fac: setting target pretesting digits to 24.62
fac: t15: 7.17
fac: t20: 1.04
fac: t25: 0.05
fac: estimated sum of completed work is t20.24
fac: work done at B1=50000: 0 curves, max work = 214 curves
fac: 188 more curves at B1=50000 needed to get to t24.62
ecm: 188/188 curves on C80, B1=50K, B2=gmp-ecm default, ETA: 0 sec
fac: setting target pretesting digits to 24.62
fac: t15: 34.02
fac: t20: 9.99
fac: t25: 0.93
fac: t30: 0.06
fac: estimated sum of completed work is t24.64

starting SIQS on c80: 33727095233132290409342213138708322681737322487170896778164145844669592994743377
fb bounds: 85%
small: 1024
SPV: 35
10bit: 96
11bit: 168
12bit: 304
13bit: 536
32k div 3: 688
14bit: 960
15bit: 1752
med: 2512
large: 14096
all: 47184
start primes
SPV: 241
10bit: 1033
11bit: 2089
12bit: 4261
13bit: 8317
32k div 3: 10987
14bit: 16451
15bit: 32789
med: 49307
large: 327571

==== sieve params ====
n = 80 digits, 265 bits
factor base: 47184 primes (max prime = 1218601)
single large prime cutoff: 109674090 (90 * pmax)
double large prime range from 41 to 49 bits
double large prime range from 1484988397201 to 296611012458251
allocating 7 large prime slices of factor base
buckets hold 2048 elements
using SSE2 enabled 32k sieve core
sieve interval: 10 blocks of size 32768
polynomial A has ~ 10 factors
using multiplier of 1
using SPV correction of 21 bits, starting at offset 35
trial factoring cutoff at 87 bits

==== sieving in progress (1 thread): 47248 relations needed ====
==== Press ctrl-c to abort and save state ====
Segmentation fault (core dumped)
redgolpe@c2018:~/yafu-tmp$
[/CODE]

bsquared 2020-03-23 15:58

I don't know why it is failing this way, but I'm using a different version of gcc and much older versions of ECM and GMP.

It may be more productive to move forward in yafu releases rather than backward because the more-recent stuff is more likely to work with more-recent versions of dependent libraries.

So, if you wouldn't mind, get a snapshot of the code at [url]https://sourceforge.net/p/yafu/code/HEAD/tree/branches/wip/[/url]
and rebuild.

While you are at it, since you have a skylake-x, build using x86_64 NFS=1 SKYLAKEX=1 USE_BMI2=1 USE_AVX2=1, as that will run 2x faster than just with SSE2.

EdH 2020-03-23 17:00

I updated gmp, ecm and msieve and now I'm crashing, too.:sad:
[code]
$ ./yafu -v -v


03/23/20 12:58:41 v1.34.5 @ math20, System/Build Info:
Using GMP-ECM 7.0.5-dev, Powered by GMP 6.2.0
detected Intel(R) Core(TM) i5-4570S CPU @ 2.90GHz
detected L1 = 32768 bytes, L2 = 6291456 bytes, CL = 64 bytes
measured cpu frequency ~= 2893.267390
using 1 random witnesses for Rabin-Miller PRP checks

===============================================================
======= Welcome to YAFU (Yet Another Factoring Utility) =======
======= bbuhrow@gmail.com =======
======= Type help at any time, or quit to quit =======
===============================================================
cached 78498 primes. pmax = 999983


>> factor(33727095233132290409342213138708322681737322487170896778164145844669592994743377)

fac: factoring 33727095233132290409342213138708322681737322487170896778164145844669592994743377
fac: using pretesting plan: normal
fac: no tune info: using qs/gnfs crossover of 95 digits
div: primes less than 10000
rho: x^2 + 3, starting 1000 iterations on C80
rho: x^2 + 2, starting 1000 iterations on C80
rho: x^2 + 1, starting 1000 iterations on C80
pm1: starting B1 = 150K, B2 = gmp-ecm default on C80
fac: setting target pretesting digits to 24.62
fac: estimated sum of completed work is t0.00
fac: work done at B1=2000: 0 curves, max work = 30 curves
fac: 30 more curves at B1=2000 needed to get to t24.62
ecm: 30/30 curves on C80, B1=2K, B2=gmp-ecm default
fac: setting target pretesting digits to 24.62
fac: t15: 1.00
fac: t20: 0.04
fac: estimated sum of completed work is t15.18
fac: work done at B1=11000: 0 curves, max work = 74 curves
fac: 74 more curves at B1=11000 needed to get to t24.62
ecm: 74/74 curves on C80, B1=11K, B2=gmp-ecm default
fac: setting target pretesting digits to 24.62
fac: t15: 7.17
fac: t20: 1.04
fac: t25: 0.05
fac: estimated sum of completed work is t20.24
fac: work done at B1=50000: 0 curves, max work = 214 curves
fac: 188 more curves at B1=50000 needed to get to t24.62
ecm: 188/188 curves on C80, B1=50K, B2=gmp-ecm default, ETA: 0 sec
fac: setting target pretesting digits to 24.62
fac: t15: 34.02
fac: t20: 9.99
fac: t25: 0.93
fac: t30: 0.06
fac: estimated sum of completed work is t24.64

starting SIQS on c80: 33727095233132290409342213138708322681737322487170896778164145844669592994743377
fb bounds: 85%
small: 1024
SPV: 35
10bit: 96
11bit: 168
12bit: 304
13bit: 536
32k div 3: 688
14bit: 960
15bit: 1752
med: 2512
large: 14096
all: 47184
start primes
SPV: 241
10bit: 1033
11bit: 2089
12bit: 4261
13bit: 8317
32k div 3: 10987
14bit: 16451
15bit: 32789
med: 49307
large: 327571

==== sieve params ====
n = 80 digits, 265 bits
factor base: 47184 primes (max prime = 1218601)
single large prime cutoff: 109674090 (90 * pmax)
double large prime range from 41 to 49 bits
double large prime range from 1484988397201 to 296611012458251
allocating 7 large prime slices of factor base
buckets hold 2048 elements
using SSE2 enabled 32k sieve core
sieve interval: 10 blocks of size 32768
polynomial A has ~ 10 factors
using multiplier of 1
using SPV correction of 21 bits, starting at offset 35
trial factoring cutoff at 87 bits

==== sieving in progress (1 thread): 47248 relations needed ====
==== Press ctrl-c to abort and save state ====
Segmentation fault (core dumped)





[/code]I'll try some other things later. . .

bsquared 2020-03-23 17:35

Ok, thanks. If you are willing, you could try (independently) older versions of ecm and gmp.
That might help pinpoint where the incompatibility occurred.

RedGolpe 2020-03-23 18:36

[QUOTE=bsquared;540611]It may be more productive to move forward in yafu releases rather than backward because the more-recent stuff is more likely to work with more-recent versions of dependent libraries.[/QUOTE]Indeed. Again, thank you for your time.

So I downloaded the wip version, modified the relevant files (note that Makefile attempts to run "gcc-7.3.0", which I edited in just "gcc"). Also, running make with the flag x86_64 did not work:[CODE]redgolpe@c2018:~/yafu$ make x86_64 NFS=1 SKYLAKEX=1 USE_BMI2=1 USE_AVX2=1
make: *** No rule to make target 'x86_64'. Stop.[/CODE]Running with all other flags it works correctly and... yay! No fragmentation faults. Please let me know if I can help in any other way.

EdH 2020-03-23 18:39

How disappointing!

I had saved, in tact, all my directories for gmp, ecm, msieve, ggnfs and yafu. (Actually I did nothing with the ggnfs directory.) I then made new installations for all.

But, now that I have gone back to the original gmp, ecm, msieve and yafu directories and reinstalled gmp and ecm via "sudo make install," the previously working yafu segmentation faults. I even tried a previous copy of a compiled yafu that I've been using in a totally different directory and it segfaults. YAFU was running fine today prior to my experiments, so it has to be something that changed today, but did not change back.

More later. . .

RedGolpe 2020-03-23 18:52

If this can be of any use, I'm fine with yafu v1.35-beta, GMP-ECM 7.0.5-dev, GMP 6.2.0, Msieve v. 1.54 (SVN 1030) compiled with gcc 7.5.0 on Ubuntu 18.04.4 LTS. I have not (yet) installed the ggnfs libraries (nor ggnfs).

EdH 2020-03-23 19:17

[QUOTE=RedGolpe;540639]If this can be of any use, I'm fine with yafu v1.35-beta, GMP-ECM 7.0.5-dev, GMP 6.2.0, Msieve v. 1.54 (SVN 1030) compiled with gcc 7.5.0 on Ubuntu 18.04.4 LTS. I have not (yet) installed the ggnfs libraries (nor ggnfs).[/QUOTE]
That sounds good that you're now up and running with the wip branch.

I will try a total upgrade with wip here, but I've got an i5 Haswell (I believe) machine, so it might be a bit different. An identical machine is running 16.04 and is fine with gmp 6.1.2 and ecm 7.0.5 with the trunk branch of YAFU. I hesitate to play with that one and break it.

For now I'll play with trying to get this broken one back to working.

Thanks for the info.

RedGolpe 2020-03-23 21:14

I spoke too soon. While I was able to run "tune" after installing the ggnfs binaries, the frag fault has returned on a C89. At least it looks like the problem is reproducible as only specific numbers seem to trigger it.[CODE]>> factor(2^295+97)
fac: factoring 63657374260452690195888927762793067532858387302060507832379389042324415617604272068231265
fac: using pretesting plan: normal
fac: using specified qs/gnfs crossover of 93 digits
fac: using specified qs/snfs crossover of 75 digits
div: primes less than 10000
Segmentation fault (core dumped)[/CODE]Even factoring 2 does not work:
[CODE]>> factor(2)
fac: factoring 2
fac: using pretesting plan: normal
fac: using specified qs/gnfs crossover of 93 digits
fac: using specified qs/snfs crossover of 75 digits
div: primes less than 10000
Segmentation fault (core dumped)[/CODE]But the problem does not seem related to small factors only: 2^274+97 fails too despite its smallest factor being a P33. On the other hand, the "tune" C80 succeeds, and also does 3528253658491813 * 790674939293910225420845660353. Contrary to the previous installation, now when the factorization does fail, it seems to do so immediately. There is no noticeable delay between my pressing ENTER and the error message.

EdH 2020-03-23 21:41

My machine is working again, with GMP 6.1.2, GMP-ECM 7.0.5, and the trunk branch of YAFU. However, AVX2 segfaults, while SSE4.1 runs fine.

Basically, I uninstalled GMP and GMP-ECM, then removed all relative directories and recreated everything (except ggnfs) based on this thread.

I will try to do some more testing tomorrow and post another update.

EdH 2020-03-23 21:47

[QUOTE=RedGolpe;540669]I spoke too soon. While I was able to run "tune" after installing the ggnfs binaries, the frag fault has returned on a C89. At least it looks like the problem is reproducible as only specific numbers seem to trigger it.[CODE]>> factor(2^295+97)
fac: factoring 63657374260452690195888927762793067532858387302060507832379389042324415617604272068231265
fac: using pretesting plan: normal
fac: using specified qs/gnfs crossover of 93 digits
fac: using specified qs/snfs crossover of 75 digits
div: primes less than 10000
Segmentation fault (core dumped)[/CODE]Even factoring 2 does not work:
[CODE]>> factor(2)
fac: factoring 2
fac: using pretesting plan: normal
fac: using specified qs/gnfs crossover of 93 digits
fac: using specified qs/snfs crossover of 75 digits
div: primes less than 10000
Segmentation fault (core dumped)[/CODE]But the problem does not seem related to small factors only: 2^274+97 fails too despite its smallest factor being a P33. On the other hand, the "tune" C80 succeeds, and also does 3528253658491813 * 790674939293910225420845660353. Contrary to the previous installation, now when the factorization does fail, it seems to do so immediately. There is no noticeable delay between my pressing ENTER and the error message.[/QUOTE]I ran the number several times with my current setup (explained in my previous post) and all ran smooth through factoring.

RedGolpe 2020-03-23 22:04

Attempting make with various flags except AVX2 fails for me. For example:
[CODE]make NFS=1 SKYLAKEX=1 USE_BMI2=1[/CODE] results in [CODE]factor/qs/SIQS.o: In function `process_poly':
/home/redgolpe/yafu/factor/qs/SIQS.c:870: undefined reference to `nextRoots_32k_avx2_small'
factor/qs/SIQS.o: In function `siqs_static_init':
/home/redgolpe/yafu/factor/qs/SIQS.c:1642: undefined reference to `nextRoots_32k_avx2'
/home/redgolpe/yafu/factor/qs/SIQS.c:1677: undefined reference to `med_sieveblock_32k_avx2'
/home/redgolpe/yafu/factor/qs/SIQS.c:1733: undefined reference to `tdiv_medprimes_32k_avx2'
/home/redgolpe/yafu/factor/qs/SIQS.c:1734: undefined reference to `resieve_medprimes_32k_avx2'
/home/redgolpe/yafu/factor/qs/SIQS.c:1685: undefined reference to `med_sieveblock_32k_sse41'
/home/redgolpe/yafu/factor/qs/SIQS.c:1650: undefined reference to `nextRoots_32k_sse41'
/home/redgolpe/yafu/factor/qs/SIQS.c:1685: undefined reference to `med_sieveblock_32k_sse41'
/home/redgolpe/yafu/factor/qs/SIQS.c:1710: undefined reference to `tdiv_medprimes_32k_avx2'
/home/redgolpe/yafu/factor/qs/SIQS.c:1734: undefined reference to `resieve_medprimes_32k_avx2'
factor/qs/SIQS.o: In function `printf':
/usr/include/x86_64-linux-gnu/bits/stdio2.h:104: undefined reference to `tdiv_medprimes_32k_avx2'
factor/qs/SIQS.o: In function `siqs_static_init':
/home/redgolpe/yafu/factor/qs/SIQS.c:1650: undefined reference to `nextRoots_32k_sse41'
collect2: error: ld returned 1 exit status
Makefile:359: recipe for target 'all' failed
make: *** [all] Error 1[/CODE]and[CODE]make NFS=1 USE_BMI2=1[/CODE]results in[CODE]In file included from /usr/lib/gcc/x86_64-linux-gnu/7/include/immintrin.h:41:0,
from include/soe.h:27,
from top/eratosthenes/primes.c:15:
/usr/lib/gcc/x86_64-linux-gnu/7/include/avxintrin.h:926:1: error: inlining failed in call to always_inline ‘_mm256_storeu_si256’: target specific option mismatch
_mm256_storeu_si256 (__m256i_u *__P, __m256i __A)
^~~~~~~~~~~~~~~~~~~
top/eratosthenes/primes.c:586:17: note: called from here
_mm256_storeu_si256((__m256i *)(&primes[GLOBAL_OFFSET + pcount]), t);
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /usr/lib/gcc/x86_64-linux-gnu/7/include/immintrin.h:41:0,
from include/soe.h:27,
from top/eratosthenes/primes.c:15:
/usr/lib/gcc/x86_64-linux-gnu/7/include/avxintrin.h:920:1: error: inlining failed in call to always_inline ‘_mm256_loadu_si256’: target specific option mismatch
_mm256_loadu_si256 (__m256i_u const *__P)
^~~~~~~~~~~~~~~~~~
top/eratosthenes/primes.c:585:25: note: called from here
__m256i t = _mm256_loadu_si256((__m256i *)(&pqueues[i][j*4]));
^
In file included from /usr/lib/gcc/x86_64-linux-gnu/7/include/immintrin.h:41:0,
from include/soe.h:27,
from top/eratosthenes/primes.c:15:
/usr/lib/gcc/x86_64-linux-gnu/7/include/avxintrin.h:926:1: error: inlining failed in call to always_inline ‘_mm256_storeu_si256’: target specific option mismatch
_mm256_storeu_si256 (__m256i_u *__P, __m256i __A)
^~~~~~~~~~~~~~~~~~~
top/eratosthenes/primes.c:586:17: note: called from here
_mm256_storeu_si256((__m256i *)(&primes[GLOBAL_OFFSET + pcount]), t);
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /usr/lib/gcc/x86_64-linux-gnu/7/include/immintrin.h:41:0,
from include/soe.h:27,
from top/eratosthenes/primes.c:15:
/usr/lib/gcc/x86_64-linux-gnu/7/include/avxintrin.h:920:1: error: inlining failed in call to always_inline ‘_mm256_loadu_si256’: target specific option mismatch
_mm256_loadu_si256 (__m256i_u const *__P)
^~~~~~~~~~~~~~~~~~
top/eratosthenes/primes.c:585:25: note: called from here
__m256i t = _mm256_loadu_si256((__m256i *)(&pqueues[i][j*4]));
^
Makefile:369: recipe for target 'top/eratosthenes/primes.o' failed
make: *** [top/eratosthenes/primes.o] Error 1[/CODE]

bsquared 2020-03-24 15:34

[QUOTE=RedGolpe;540634]Indeed. Again, thank you for your time.

So I downloaded the wip version, modified the relevant files (note that Makefile attempts to run "gcc-7.3.0", which I edited in just "gcc"). Also, running make with the flag x86_64 did not work:[CODE]redgolpe@c2018:~/yafu$ make x86_64 NFS=1 SKYLAKEX=1 USE_BMI2=1 USE_AVX2=1
make: *** No rule to make target 'x86_64'. Stop.[/CODE]Running with all other flags it works correctly and... yay! No fragmentation faults. Please let me know if I can help in any other way.[/QUOTE]

Good! Glad that worked out. Even more incentive (as if I needed it) for me to get the wip branch merged back into trunk. Based on this and possibly EdH's trials it appears that trunk might no longer be compatible with the most-recent gmp or ecm somehow. EdH if you produce data supporting this it would be cool to hear it.

RedGolpe 2020-03-24 15:53

[QUOTE=bsquared;540756]Glad that worked out.[/QUOTE]Actually it didn't as the seg fault reappeared with new factorizations: see the following messages. As I said, it even fails with factor(2) which should be a good target for debugging: if I can help in any way, please let me know.

bsquared 2020-03-24 16:05

[QUOTE=RedGolpe;540758]Actually it didn't as the seg fault reappeared with new factorizations: see the following messages. As I said, it even fails with factor(2) which should be a good target for debugging: if I can help in any way, please let me know.[/QUOTE]

Oh, sorry. Didn't refresh the page before responding. I will take a look.

EdH 2020-03-24 19:09

This is repeatable:

If I uninstall GMP 6.1.2 and install 6.2.0, YAFU fails:
[code]
$ ./yafu -v -v

03/24/20 14:48:31 v1.34.5 @ math20, System/Build Info:
Using GMP-ECM 7.0.5-dev, Powered by GMP 6.2.0
detected Intel(R) Core(TM) i5-4570S CPU @ 2.90GHz
detected L1 = 32768 bytes, L2 = 6291456 bytes, CL = 64 bytes
measured cpu frequency ~= 2893.258220
using 1 random witnesses for Rabin-Miller PRP checks

===============================================================
======= Welcome to YAFU (Yet Another Factoring Utility) =======
======= bbuhrow@gmail.com =======
======= Type help at any time, or quit to quit =======
===============================================================
cached 78498 primes. pmax = 999983

>> factor(33727095233132290409342213138708322681737322487170896778164145844669592994743377)

fac: factoring 33727095233132290409342213138708322681737322487170896778164145844669592994743377
fac: using pretesting plan: normal
fac: no tune info: using qs/gnfs crossover of 95 digits
div: primes less than 10000
rho: x^2 + 3, starting 1000 iterations on C80
rho: x^2 + 2, starting 1000 iterations on C80
rho: x^2 + 1, starting 1000 iterations on C80
pm1: starting B1 = 150K, B2 = gmp-ecm default on C80
fac: setting target pretesting digits to 24.62
fac: estimated sum of completed work is t0.00
fac: work done at B1=2000: 0 curves, max work = 30 curves
fac: 30 more curves at B1=2000 needed to get to t24.62
ecm: 32/32 curves on C80, B1=2K, B2=gmp-ecm default
fac: setting target pretesting digits to 24.62
fac: t15: 1.07
fac: t20: 0.04
fac: estimated sum of completed work is t15.19
fac: work done at B1=11000: 0 curves, max work = 74 curves
fac: 74 more curves at B1=11000 needed to get to t24.62
ecm: 76/76 curves on C80, B1=11K, B2=gmp-ecm default
fac: setting target pretesting digits to 24.62
fac: t15: 7.40
fac: t20: 1.06
fac: t25: 0.05
fac: estimated sum of completed work is t20.25
fac: work done at B1=50000: 0 curves, max work = 214 curves
fac: 187 more curves at B1=50000 needed to get to t24.62
ecm: 188/188 curves on C80, B1=50K, B2=gmp-ecm default, ETA: 0 sec
fac: setting target pretesting digits to 24.62
fac: t15: 34.26
fac: t20: 10.02
fac: t25: 0.93
fac: t30: 0.06
fac: estimated sum of completed work is t24.64

starting SIQS on c80: 33727095233132290409342213138708322681737322487170896778164145844669592994743377
fb bounds: 85%
small: 1024
SPV: 35
10bit: 96
11bit: 168
12bit: 304
13bit: 536
32k div 3: 688
14bit: 960
15bit: 1752
med: 2512
large: 14096
all: 47184
start primes
SPV: 241
10bit: 1033
11bit: 2089
12bit: 4261
13bit: 8317
32k div 3: 10987
14bit: 16451
15bit: 32789
med: 49307
large: 327571

==== sieve params ====
n = 80 digits, 265 bits
factor base: 47184 primes (max prime = 1218601)
single large prime cutoff: 109674090 (90 * pmax)
double large prime range from 41 to 49 bits
double large prime range from 1484988397201 to 296611012458251
allocating 7 large prime slices of factor base
buckets hold 2048 elements
using SSE2 enabled 32k sieve core
sieve interval: 10 blocks of size 32768
polynomial A has ~ 10 factors
using multiplier of 1
using SPV correction of 21 bits, starting at offset 35
trial factoring cutoff at 87 bits

==== sieving in progress ( 4 threads): 47248 relations needed ====
==== Press ctrl-c to abort and save state ====
Segmentation fault (core dumped)
[/code]If I uninstall 6.2.0 and reinstall 6.1.2, YAFU succeeds:
[code]
$ ./yafu -v -v

03/24/20 14:54:44 v1.34.5 @ math20, System/Build Info:
Using GMP-ECM 7.0.5-dev, Powered by GMP 6.1.2
detected Intel(R) Core(TM) i5-4570S CPU @ 2.90GHz
detected L1 = 32768 bytes, L2 = 6291456 bytes, CL = 64 bytes
measured cpu frequency ~= 2893.276310
using 1 random witnesses for Rabin-Miller PRP checks

===============================================================
======= Welcome to YAFU (Yet Another Factoring Utility) =======
======= bbuhrow@gmail.com =======
======= Type help at any time, or quit to quit =======
===============================================================
cached 78498 primes. pmax = 999983

>> factor(33727095233132290409342213138708322681737322487170896778164145844669592994743377)

fac: factoring 33727095233132290409342213138708322681737322487170896778164145844669592994743377
fac: using pretesting plan: normal
fac: no tune info: using qs/gnfs crossover of 95 digits
div: primes less than 10000
rho: x^2 + 3, starting 1000 iterations on C80
rho: x^2 + 2, starting 1000 iterations on C80
rho: x^2 + 1, starting 1000 iterations on C80
pm1: starting B1 = 150K, B2 = gmp-ecm default on C80
fac: setting target pretesting digits to 24.62
fac: estimated sum of completed work is t0.00
fac: work done at B1=2000: 0 curves, max work = 30 curves
fac: 30 more curves at B1=2000 needed to get to t24.62
ecm: 32/32 curves on C80, B1=2K, B2=gmp-ecm default
fac: setting target pretesting digits to 24.62
fac: t15: 1.07
fac: t20: 0.04
fac: estimated sum of completed work is t15.19
fac: work done at B1=11000: 0 curves, max work = 74 curves
fac: 74 more curves at B1=11000 needed to get to t24.62
ecm: 76/76 curves on C80, B1=11K, B2=gmp-ecm default
fac: setting target pretesting digits to 24.62
fac: t15: 7.40
fac: t20: 1.06
fac: t25: 0.05
fac: estimated sum of completed work is t20.25
fac: work done at B1=50000: 0 curves, max work = 214 curves
fac: 187 more curves at B1=50000 needed to get to t24.62
ecm: 188/188 curves on C80, B1=50K, B2=gmp-ecm default, ETA: 0 sec
fac: setting target pretesting digits to 24.62
fac: t15: 34.26
fac: t20: 10.02
fac: t25: 0.93
fac: t30: 0.06
fac: estimated sum of completed work is t24.64

starting SIQS on c80: 33727095233132290409342213138708322681737322487170896778164145844669592994743377
fb bounds: 85%
small: 1024
SPV: 35
10bit: 96
11bit: 168
12bit: 304
13bit: 536
32k div 3: 688
14bit: 960
15bit: 1752
med: 2512
large: 14096
all: 47184
start primes
SPV: 241
10bit: 1033
11bit: 2089
12bit: 4261
13bit: 8317
32k div 3: 10987
14bit: 16451
15bit: 32789
med: 49307
large: 327571

==== sieve params ====
n = 80 digits, 265 bits
factor base: 47184 primes (max prime = 1218601)
single large prime cutoff: 109674090 (90 * pmax)
double large prime range from 41 to 49 bits
double large prime range from 1484988397201 to 296611012458251
allocating 7 large prime slices of factor base
buckets hold 2048 elements
using SSE2 enabled 32k sieve core
sieve interval: 10 blocks of size 32768
polynomial A has ~ 10 factors
using multiplier of 1
using SPV correction of 21 bits, starting at offset 35
trial factoring cutoff at 87 bits

==== sieving in progress ( 4 threads): 47248 relations needed ====
==== Press ctrl-c to abort and save state ====
36150 rels found: 16473 full + 19677 from 343453 partial, (13629.81 rels/sec)

sieving required 64261 total polynomials
trial division touched 6551746 sieve locations out of 42114088960
squfof: 0 failures, 218488 attempts, 1769438 outside range, 1239973 prp, 207029 useful
total reports = 6551746, total surviving reports = 4092848
total blocks sieved = 1285220,avg surviving reports per block = 3.18
QS elapsed time = 30.9348 seconds.

==== post processing stage (msieve-1.38) ====
begin with 424829 relations
reduce to 90315 relations in 9 passes
attempting to read 90315 relations
recovered 90315 relations
recovered 48552 polynomials
attempting to build 48489 cycles
found 48489 cycles in 4 passes
distribution of cycle lengths:
length 1 : 19202
length 2 : 14652
length 3 : 8320
length 4 : 3749
length 5 : 1543
length 6 : 660
length 7 : 251
length 9+: 112
largest cycle: 12 relations
matrix is 47184 x 48489 (8.7 MB) with weight 1892112 (39.02/col)
sparse part has weight 1892112 (39.02/col)
filtering completed in 4 passes
matrix is 36883 x 36947 (6.8 MB) with weight 1474089 (39.90/col)
sparse part has weight 1474089 (39.90/col)
saving the first 48 matrix rows for later
matrix is 36835 x 36947 (5.7 MB) with weight 1235125 (33.43/col)
sparse part has weight 1114954 (30.18/col)
matrix includes 64 packed rows
using block size 14778 for processor cache size 6144 kB
commencing Lanczos iteration
memory use: 4.9 MB
lanczos halted after 584 iterations (dim = 36831)
recovered 14 nontrivial dependencies
Lanczos elapsed time = 2.7926 seconds.
Sqrt elapsed time = 0.0323 seconds.
SIQS elapsed time = 33.7602 seconds.
pretesting / qs ratio was 0.19
Total factoring time = 40.3479 seconds

***factors found***

P40 = 3104758941649954796179093599696620654807
P41 = 10863031838216972467057273004441996191511

ans = 1

>>
[/code]I did not reinstall GMP-ECM or recompile msieve. Both are latest (7.0.5 and 1030), though they were compiled with GMP 6.1.2 installed.

I compiled YAFU with no options; (make x86_64).

I didn't try the wip branch today, but I haven't been able to compile it at all on this machine for quite some time. I will try to play with this some more soon.

bsquared 2020-03-24 19:48

[QUOTE=EdH;540769]This is repeatable:

If I uninstall GMP 6.1.2 and install 6.2.0, YAFU fails:

I did not reinstall GMP-ECM or recompile msieve. Both are latest (7.0.5 and 1030), though they were compiled with GMP 6.1.2 installed.

I compiled YAFU with no options; (make x86_64).

I didn't try the wip branch today, but I haven't been able to compile it at all on this machine for quite some time. I will try to play with this some more soon.[/QUOTE]

Thank you very much. Looks like something is incompatible with gmp 6.2.0. I would imagine whatever this is, is there no matter the yafu version. So until I figure out what changed, going back to gmp 6.1.2 would probably be the way to go.

RedGolpe 2020-03-24 20:05

[QUOTE=bsquared;540774]Thank you very much. Looks like something is incompatible with gmp 6.2.0. I would imagine whatever this is, is there no matter the yafu version. So until I figure out what changed, going back to gmp 6.1.2 would probably be the way to go.[/QUOTE]Does factor(2) even invoke GMP?

bsquared 2020-03-24 20:11

[QUOTE=RedGolpe;540779]Does factor(2) even invoke GMP?[/QUOTE]

All inputs get converted into mpz_t's (GMP's signed-integer data type). And it has no way of knowing the input is a '2' except by running trial division (on the mpt_z) or is_probab_prime_p() (on the mpz_t).

So, yes.

RedGolpe 2020-03-24 20:30

I tried factor(p) on primes of various size on my almost-working build. It apparently segfaults for all primes below 10000 and succeeds for all primes above. It also seems to segfault for all factorizations whose input factors in such small primes, and not otherwise. That would explain why "tune" succeeds, that being focused on products of two large primes. This hypothesis is also consistent with my previous segfaults, including the C89 divisible by 5.

bsquared 2020-03-24 20:33

[QUOTE=RedGolpe;540785]I tried factor(p) on primes of various size on my almost-working build. It apparently segfaults for all primes below 10000 and succeeds for all primes above. It also seems to segfault for all factorizations whose input factors in such small primes, and not otherwise. That would explain why "tune" succeeds, that being focused on products of two large primes. This hypothesis is also consistent with my previous segfaults, including the C89 divisible by 5.[/QUOTE]

Thank you, this is extremely helpful! It seems to isolate the problem to when trial division succeeds.

RedGolpe 2020-03-24 20:53

I just discovered that factor(2^308+97) also fails but has no factors below 10000. However, it fails with[code]>> factor(2^308+97)
fac: factoring 521481209941628438084722096232800809229175908778479680162851955034721612739414196782949728353
fac: using pretesting plan: normal
fac: using specified qs/gnfs crossover of 93 digits
fac: using specified qs/snfs crossover of 75 digits
div: primes less than 10000
fmt: 1000000 iterations
rho: x^2 + 3, starting 200 iterations on C93
rho: found prp6 factor = 785033
rho: x^2 + 3, starting 200 iterations on C87
rho: x^2 + 2, starting 200 iterations on C87
rho: x^2 + 1, starting 200 iterations on C87
nfs: searching for brent special forms...
nfs: input divides 2^308 + 97
Segmentation fault (core dumped)[/code]while factor(2) fails with
[code]>> factor(2)
fac: factoring 2
fac: using pretesting plan: normal
fac: using specified qs/gnfs crossover of 93 digits
fac: using specified qs/snfs crossover of 75 digits
div: primes less than 10000
Segmentation fault (core dumped)[/code]

EdH 2020-03-24 20:58

For the wip branch I have to change the "CC = gcc-7.3.0" to "CC = gcc" since my gcc is "gcc version 7.5.0 (Ubuntu 7.5.0-3ubuntu1~18.04)."

With the wip branch I get the following with either version of GMP:
[code]
factor/rho.o: In function `_trail_zcnt64':
/home/math20/Math/yafuwip2/include/arith.h:102: undefined reference to `_BitScanForward64'
/home/math20/Math/yafuwip2/include/arith.h:102: undefined reference to `_BitScanForward64'
factor/squfof.o: In function `_lead_zcnt64':
/home/math20/Math/yafuwip2/include/arith.h:110: undefined reference to `_BitScanReverse64'
arith/arith3.o: In function `_trail_zcnt64':
/home/math20/Math/yafuwip2/include/arith.h:102: undefined reference to `_BitScanForward64'
/home/math20/Math/yafuwip2/include/arith.h:102: undefined reference to `_BitScanForward64'
/home/math20/Math/yafuwip2/include/arith.h:102: undefined reference to `_BitScanForward64'
arith/monty.o: In function `mulmod128':
/home/math20/Math/yafuwip2/arith/monty.c:416: undefined reference to `_umul128'
/home/math20/Math/yafuwip2/arith/monty.c:423: undefined reference to `_umul128'
/home/math20/Math/yafuwip2/arith/monty.c:423: undefined reference to `_addcarry_u64'
/home/math20/Math/yafuwip2/arith/monty.c:424: undefined reference to `_addcarry_u64'
/home/math20/Math/yafuwip2/arith/monty.c:428: undefined reference to `_umul128'
/home/math20/Math/yafuwip2/arith/monty.c:428: undefined reference to `_addcarry_u64'
/home/math20/Math/yafuwip2/arith/monty.c:429: undefined reference to `_umul128'
/home/math20/Math/yafuwip2/arith/monty.c:429: undefined reference to `_addcarry_u64'
/home/math20/Math/yafuwip2/arith/monty.c:430: undefined reference to `_addcarry_u64'
/home/math20/Math/yafuwip2/arith/monty.c:431: undefined reference to `_addcarry_u64'
/home/math20/Math/yafuwip2/arith/monty.c:440: undefined reference to `_umul128'
/home/math20/Math/yafuwip2/arith/monty.c:440: undefined reference to `_addcarry_u64'
/home/math20/Math/yafuwip2/arith/monty.c:441: undefined reference to `_addcarry_u64'
/home/math20/Math/yafuwip2/arith/monty.c:449: undefined reference to `_umul128'
/home/math20/Math/yafuwip2/arith/monty.c:449: undefined reference to `_addcarry_u64'
/home/math20/Math/yafuwip2/arith/monty.c:450: undefined reference to `_addcarry_u64'
/home/math20/Math/yafuwip2/arith/monty.c:455: undefined reference to `_umul128'
/home/math20/Math/yafuwip2/arith/monty.c:455: undefined reference to `_addcarry_u64'
/home/math20/Math/yafuwip2/arith/monty.c:456: undefined reference to `_umul128'
/home/math20/Math/yafuwip2/arith/monty.c:456: undefined reference to `_addcarry_u64'
/home/math20/Math/yafuwip2/arith/monty.c:457: undefined reference to `_addcarry_u64'
/home/math20/Math/yafuwip2/arith/monty.c:458: undefined reference to `_addcarry_u64'
top/eratosthenes/primes.o: In function `_trail_zcnt64':
/home/math20/Math/yafuwip2/include/arith.h:102: undefined reference to `_BitScanForward64'
collect2: error: ld returned 1 exit status
Makefile:359: recipe for target 'all' failed
make: *** [all] Error 1
[/code]If I compile the trunk branch of YAFU with USE_AVX2=1, I also get a segfault in SIQS:
[code]
$ ./yafu -v -v


03/24/20 16:47:09 v1.34.5 @ math20, System/Build Info:
Using GMP-ECM 7.0.5-dev, Powered by GMP 6.1.2
detected Intel(R) Core(TM) i5-4570S CPU @ 2.90GHz
detected L1 = 32768 bytes, L2 = 6291456 bytes, CL = 64 bytes
measured cpu frequency ~= 2893.255040
using 1 random witnesses for Rabin-Miller PRP checks

===============================================================
======= Welcome to YAFU (Yet Another Factoring Utility) =======
======= bbuhrow@gmail.com =======
======= Type help at any time, or quit to quit =======
===============================================================
cached 78498 primes. pmax = 999983


>> factor(33727095233132290409342213138708322681737322487170896778164145844669592994743377)

fac: factoring 33727095233132290409342213138708322681737322487170896778164145844669592994743377
fac: using pretesting plan: normal
fac: no tune info: using qs/gnfs crossover of 95 digits
div: primes less than 10000
rho: x^2 + 3, starting 1000 iterations on C80
rho: x^2 + 2, starting 1000 iterations on C80
rho: x^2 + 1, starting 1000 iterations on C80
pm1: starting B1 = 150K, B2 = gmp-ecm default on C80
fac: setting target pretesting digits to 24.62
fac: estimated sum of completed work is t0.00
fac: work done at B1=2000: 0 curves, max work = 30 curves
fac: 30 more curves at B1=2000 needed to get to t24.62
ecm: 32/32 curves on C80, B1=2K, B2=gmp-ecm default
fac: setting target pretesting digits to 24.62
fac: t15: 1.07
fac: t20: 0.04
fac: estimated sum of completed work is t15.19
fac: work done at B1=11000: 0 curves, max work = 74 curves
fac: 74 more curves at B1=11000 needed to get to t24.62
ecm: 76/76 curves on C80, B1=11K, B2=gmp-ecm default
fac: setting target pretesting digits to 24.62
fac: t15: 7.40
fac: t20: 1.06
fac: t25: 0.05
fac: estimated sum of completed work is t20.25
fac: work done at B1=50000: 0 curves, max work = 214 curves
fac: 187 more curves at B1=50000 needed to get to t24.62
ecm: 188/188 curves on C80, B1=50K, B2=gmp-ecm default, ETA: 0 sec
fac: setting target pretesting digits to 24.62
fac: t15: 34.26
fac: t20: 10.02
fac: t25: 0.93
fac: t30: 0.06
fac: estimated sum of completed work is t24.64

starting SIQS on c80: 33727095233132290409342213138708322681737322487170896778164145844669592994743377
fb bounds: 85%
small: 1024
SPV: 35
10bit: 96
11bit: 168
12bit: 304
13bit: 536
32k div 3: 688
14bit: 960
15bit: 1752
med: 2512
large: 14096
all: 47184
start primes
SPV: 241
10bit: 1033
11bit: 2089
12bit: 4261
13bit: 8317
32k div 3: 10987
14bit: 16451
15bit: 32789
med: 49307
large: 327571

==== sieve params ====
n = 80 digits, 265 bits
factor base: 47184 primes (max prime = 1218601)
single large prime cutoff: 109674090 (90 * pmax)
double large prime range from 41 to 49 bits
double large prime range from 1484988397201 to 296611012458251
allocating 7 large prime slices of factor base
buckets hold 2048 elements
using AVX2 enabled 32k sieve core
sieve interval: 10 blocks of size 32768
polynomial A has ~ 10 factors
using multiplier of 1
using SPV correction of 21 bits, starting at offset 35
trial factoring cutoff at 87 bits

==== sieving in progress ( 4 threads): 47248 relations needed ====
==== Press ctrl-c to abort and save state ====
Segmentation fault (core dumped)
[/code]USE_SSE41=1 works fine.

bsquared 2020-03-26 16:00

gmp-6.2.0 has a new lazy allocation strategy that makes it even more dangerous to access mpz_t internals directly (like _mp_d). I was bad and had some code accessing internals... that's what was causing the problem.

Should be fixed now in /branches/wip r382. I tested the following makes in gcc 7.3.0

NFS=1 USE_SSE41=1
NFS=1 USE_AVX2=1
NFS=1 USE_AVX2=1 USE_BMI2=1
NFS=1 USE_AVX2=1 SKYLAKEX=1

RedGolpe 2020-03-26 16:21

I tried again but I can't manage "make" to work, apparently with any combination of the flags NFS=1 SKYLAKEX=1 USE_BMI2=1 USE_AVX2=1. At least, all tries with various flags seem to produce the same error:[CODE]redgolpe@c2018:~/yafu$ make NFS=1 SKYLAKEX=1 USE_BMI2=1 USE_AVX2=1
gcc -g -DUSE_BMI2 -DUSE_AVX2 -DUSE_AVX512F -DUSE_AVX512BW -march=skylake-avx512 -mbmi2 -mbmi -DUSE_BMI2 -DUSE_AVX2 -DUSE_SSE41 -mavx2 -DUSE_SSE41 -m64 -msse4.1 -DUSE_NFS -O2 -fomit-frame-pointer -Wall -I. -Iinclude -Itop/aprcl -Itop/ -I../msieve/zlib -I../gmp/include -I../gmp-ecm/include/ -c -o factor/qs/msieve/lanczos.o factor/qs/msieve/lanczos.c
In file included from include/qs.h:30:0,
from factor/qs/msieve/lanczos.c:20:
include/cofactorize.h:1:1: error: version control conflict marker in file
<<<<<<< .mine
^~~~~~~
include/cofactorize.h:228:1: error: expected identifier or ‘(’ before ‘||’ token
||||||| .r0
^~
In file included from factor/qs/msieve/lanczos.c:20:0:
include/qs.h:182:3: warning: data definition has no type or storage class
} qs_params;
^~~~~~~~~
include/qs.h:182:3: warning: type defaults to ‘int’ in declaration of ‘qs_params’ [-Wimplicit-int]
include/qs.h:824:1: error: unknown type name ‘qs_params’; did you mean ‘tiny_qs_params’?
qs_params sieve_params;
^~~~~~~~~
tiny_qs_params
factor/qs/msieve/lanczos.c: In function ‘yafu_block_lanczos_core’:
factor/qs/msieve/lanczos.c:1245:5: warning: null argument where non-null required (argument 1) [-Wnonnull]
fprintf(NULL, "linear algebra completed %u "
^~~~~~~
factor/qs/msieve/lanczos.c: In function ‘yafu_read_lanczos_state’:
factor/qs/msieve/lanczos.c:854:2: warning: ignoring return value of ‘fread’, declared with attribute warn_unused_result [-Wunused-result]
fread(&read_n, sizeof(uint32), (size_t)1, dump_fp);
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Makefile:369: recipe for target 'factor/qs/msieve/lanczos.o' failed
make: *** [factor/qs/msieve/lanczos.o] Error 1[/CODE]

bsquared 2020-03-26 16:43

Sorry - sometimes svn is frustrating. I removed the SVN-inserted conflict marker junk... Try it now.

RedGolpe 2020-03-26 17:03

factor(2) now works but factor(2^308+97) still segfaults.[CODE]redgolpe@c2018:~/yafu$ ./yafu


03/26/20 18:00:43 v1.35-beta @ c2018, System/Build Info:
Using GMP-ECM 7.0.5-dev, Powered by GMP 6.2.0
detected Intel(R) Core(TM) i7-7800X CPU @ 3.50GHz
detected L1 = 32768 bytes, L2 = 8650752 bytes, CL = 64 bytes
measured cpu frequency ~= 3503.994240
using 1 random witnesses for Rabin-Miller PRP checks

===============================================================
======= Welcome to YAFU (Yet Another Factoring Utility) =======
======= bbuhrow@gmail.com =======
======= Type help at any time, or quit to quit =======
===============================================================
cached 78498 primes. pmax = 999983


>> factor(2)
fac: factoring 2
fac: using pretesting plan: normal
fac: using specified qs/gnfs crossover of 93 digits
fac: using specified qs/snfs crossover of 75 digits
div: primes less than 10000
Total factoring time = 0.0047 seconds


***factors found***

P1 = 2
1
>> factor(2^308+97)
fac: factoring 521481209941628438084722096232800809229175908778479680162851955034721612739414196782949728353
fac: using pretesting plan: normal
fac: using specified qs/gnfs crossover of 93 digits
fac: using specified qs/snfs crossover of 75 digits
div: primes less than 10000
fmt: 1000000 iterations
rho: x^2 + 3, starting 200 iterations on C93
rho: x^2 + 3, starting 200 iterations on C87
rho: x^2 + 2, starting 200 iterations on C87
rho: x^2 + 1, starting 200 iterations on C87
nfs: searching for brent special forms...
nfs: input divides 2^308 + 97
Segmentation fault (core dumped)
redgolpe@c2018:~/yafu$[/CODE]

RedGolpe 2020-03-26 17:39

SIQS seems to work for any number. Segfaults seem confined to composites for which NFS is invoked.[CODE]>> siqs(2^308+97)

starting SIQS on c93:
[...]
***factors found***

P6 = 785033
P41 = 21544259627996485962520784751544229716373

***co-factor***
C47 = 30833240972859993737296999866502074697103820917

ans = 24205111660647199463571475696199723205691503845935261

>>[/CODE]

bsquared 2020-03-26 17:56

[QUOTE=RedGolpe;540946]SIQS seems to work for any number. Segfaults seem confined to composites for which NFS is invoked.[CODE]>> siqs(2^308+97)

starting SIQS on c93:
[...]
***factors found***

P6 = 785033
P41 = 21544259627996485962520784751544229716373

***co-factor***
C47 = 30833240972859993737296999866502074697103820917

ans = 24205111660647199463571475696199723205691503845935261

>>[/CODE][/QUOTE]

I've traced this back into the msieve library. Seems msieve has the same problem yafu did w.r.t gmp-6.2.0. I don't have developer privileges for the msieve repository but making this change locally and then recompiling (first msieve, then yafu) should work (haven't tested yet - busy with other stuff and won't be able to for a little bit).

in include\gmp_xface.h replace this code (starting at line 50 in my version)
[CODE]static INLINE void uint64_2gmp(uint64 src, mpz_t dest) {

#if GMP_LIMB_BITS == 64
dest->_mp_d[0] = src;
dest->_mp_size = (src ? 1 : 0);
#else
/* mpz_import is terribly slow */
mpz_set_ui(dest, (uint32)(src >> 32));
mpz_mul_2exp(dest, dest, 32);
mpz_add_ui(dest, dest, (uint32)src);
#endif
}
[/CODE]


with this code:

[CODE]
static INLINE void uint64_2gmp(uint64 src, mpz_t dest) {
#if GMP_LIMB_BITS == 64
mpz_set_ui(dest, src);
#else
/* mpz_import is terribly slow */
mpz_set_ui(dest, (uint32)(src >> 32));
mpz_mul_2exp(dest, dest, 32);
mpz_add_ui(dest, dest, (uint32)src);
#endif
}
[/CODE]

RedGolpe 2020-03-26 18:35

It looks like it's working, no errors on any input so far with the msieve patch. I assume you have means to directly contact the author about the needed fix. Thank you for the time you spend on this project, and let me know if you need a beta tester :wink:

bsquared 2020-03-26 19:00

[QUOTE=RedGolpe;540951]It looks like it's working, no errors on any input so far with the msieve patch. I assume you have means to directly contact the author about the needed fix. Thank you for the time you spend on this project, and let me know if you need a beta tester :wink:[/QUOTE]

Yep, I can let him (jasonp) know.

Thank you very much for your reports and testing! This is how these projects get better.

:smile:

EdH 2020-03-26 20:38

The wip branch appears to be working 100% for me here now with GMP 6.2.0.

Thanks much!

RedGolpe 2020-03-27 15:28

Results on Ubuntu 18.04.4 with yafu v1.35-beta, GMP-ECM 7.0.5-dev, GMP 6.2.0, Msieve 1.54 (patched) of factor(2^308+97):

Intel(R) Core(TM) i7-7800X CPU @ 3.50GHz (skylake), 12 logical processors, 32 GB RAM:
12.3395 seconds


Intel(R) Core(TM) i7-5820K CPU @ 3.30GHz (haswell), 12 logical processors, 8 GB RAM:
14.9494 seconds

Intel(R) Core(TM) i7-3820 CPU @ 3.60GHz (sandy bridge), 8 logical processors, 8 GB RAM:
21.1907 seconds

Mr. Odd 2020-07-01 19:48

Ed - your guides are a God-send, thanks so much for writing them up and, more importantly, keeping them current!

I installed everything without a hitch - except Yafu.

Not an issue, just looking for clarification:
I couldn't understand the "Using a text editor, open the Makefile file and make the following edit:" section (was I supposed to just change the highlighted text?) so I just replaced the entire 'ifeq ($(NFS),1)' clause in the Makefile with what you posted, that seemed to work fine.

The one edit I had to make that you didn't list:
I got an error executing the Makefile because it couldn't find the command 'gcc-7.2.1' so I changed it to 'gcc' and that did the trick.


After many years, it's great to be current with my factoring tools. On my newly built 32 thread AMD 3950X Ubuntu 19.10 system, I factored 2^308+97 in 5.0s. :smile: On to bigger fish!

EdH 2020-07-01 20:47

[QUOTE=Mr. Odd;549556]Ed - your guides are a God-send, thanks so much for writing them up and, more importantly, keeping them current!

I installed everything without a hitch - except Yafu.

Not an issue, just looking for clarification:
I couldn't understand the "Using a text editor, open the Makefile file and make the following edit:" section (was I supposed to just change the highlighted text?) so I just replaced the entire 'ifeq ($(NFS),1)' clause in the Makefile with what you posted, that seemed to work fine.

The one edit I had to make that you didn't list:
I got an error executing the Makefile because it couldn't find the command 'gcc-7.2.1' so I changed it to 'gcc' and that did the trick.


After many years, it's great to be current with my factoring tools. On my newly built 32 thread AMD 3950X Ubuntu 19.10 system, I factored 2^308+97 in 5.0s. :smile: On to bigger fish![/QUOTE]Thanks for the report and reminding me of the GCC edit. I actually just comment out that line. It's only necessary to edit it for the wip version

Yes, the bold line(s) is(are) the only change(s) necessary.

I wonder about your gcc-7.2.1, though. That seems old. It's been 7.3.0 for many revisions.

Mr. Odd 2020-07-01 21:05

I'm using the wip version.

Interesting about the bold edit - the text in my make file looked nothing like what you quoted.

Sorry, my mistake, it was gcc-7.3.0 and my system now has gcc-9.2.1, so I mixed them up.

Eh, all's well that end's well.

EdH 2020-07-01 21:20

[QUOTE=Mr. Odd;549561]I'm using the wip version.

Interesting about the bold edit - the text in my make file looked nothing like what you quoted.

Sorry, my mistake, it was gcc-7.3.0 and my system now has gcc-9.2.1, so I mixed them up.

Eh, all's well that end's well.[/QUOTE]
B[SUP]2[/SUP] has probably changed it again! He does that, which is a good thing. It helps me keep on my toes and it shows he's actively improving the YAFU package. The yafu.ini file has changed in a big way just recently.

sivandahan7 2020-08-10 15:08

Hi,
Thank you for installation instructions,
It's looking good and working correctly !
But it's not working as I expecting,
According to "lscpu" command, I have 8 Cpu(s) and 2 threads per cpu,
Together it's 16 threads (this is the number I put on threads field of "yafu.ini" file),
According to "top" and "monitor" Ubuntu tools,
Actually I see yafu running on one cpu with 100% usage,
What about other cpus and why yafu does no use them ?
Do I miss something ?
Thanks a lot,

EdH 2020-08-10 21:59

[QUOTE=sivandahan7;553127]Hi,
Thank you for installation instructions,
It's looking good and working correctly !
But it's not working as I expecting,
According to "lscpu" command, I have 8 Cpu(s) and 2 threads per cpu,
Together it's 16 threads (this is the number I put on threads field of "yafu.ini" file),
According to "top" and "monitor" Ubuntu tools,
Actually I see yafu running on one cpu with 100% usage,
What about other cpus and why yafu does no use them ?
Do I miss something ?
Thanks a lot,[/QUOTE]Thanks for the feedback. Only a couple things come to mind:

1. Make sure the yafu.ini file is in the directory that you are using to call YAFU.

2. Make sure that the threads line isn't commented out with a "%" symbol.

You might also try adding "-threads 16" (without quotes) to the command line and see if that works.

kruoli 2020-08-11 08:15

Please try running something with SIQS. It should look something like this: [c]==== sieving in progress ( 16 threads): 82576 relations needed ====[/c]
In some steps, yafu does not use more then 1 thread. You might have to add [c]prefer_gmpecm[/c] to the ini-file and make sure the GMP-ECM binary's location is set correctly.

sivandahan7 2020-08-13 08:41

Thanks,
Now it's working correctly,
Unfortunately, I'm getting "Segmentation fault" while I try to factor 512 bit number,
Please see the following log:
[code]
applying option threads=16
applying option aprcl_p=500
applying option aprcl_d=200
applying option ggnfs_dir=/home/user/Desktop/yafu/Math/ggnfs/bin/
applying option B1ecm=11000
applying option ecm_path=/usr/local/bin/ecm
applying option B1pm1=20000
applying option tune_info=Intel(R) Xeon(R) Gold 5122 CPU @ 3.60GHz,LINUX64,1.59078e-05,0.196092,0.299688,0.0999245,102.36,42
applying option tune_info=Intel(R) Core(TM) i7-8700T CPU @ 2.40GHz,LINUX64,7.5902e-05,0.182524,0.621489,0.0992485,108.201,2399.99


08/12/20 12:09:54 v1.35-beta @ user-virtual-machine, System/Build Info:
Using GMP-ECM 7.0.5-dev, Powered by GMP 6.2.0
detected Intel(R) Core(TM) i7-8700T CPU @ 2.40GHz
detected L1 = 32768 bytes, L2 = 12582912 bytes, CL = 64 bytes
measured cpu frequency ~= 2399.983520
using 1 random witnesses for Rabin-Miller PRP checks

===============================================================
======= Welcome to YAFU (Yet Another Factoring Utility) =======
======= [EMAIL="bbuhrow@gmail.com"]bbuhrow@gmail.com[/EMAIL] =======
======= Type help at any time, or quit to quit =======
===============================================================
cached 664579 primes. pmax = 9999991


>>
>> factor(7481941016577287348099705569618881449446613674024887958060868042015962068162606112448282331657456711392343427317702862486293906907971066705762469196914429)
fac: factoring 7481941016577287348099705569618881449446613674024887958060868042015962068162606112448282331657456711392343427317702862486293906907971066705762469196914429
fac: using pretesting plan: normal
fac: using tune info for qs/gnfs crossover
div: primes less than 10000
fmt: 1000000 iterations
rho: x^2 + 3, starting 1000 iterations on C154
rho: x^2 + 2, starting 1000 iterations on C154
rho: x^2 + 1, starting 1000 iterations on C154
nfs: searching for brent special forms...
nfs: searching for homogeneous cunningham special forms...
nfs: searching for XYYXF special forms...
nfs: couldn't find special form
pm1: starting B1 = 150K, B2 = gmp-ecm default on C154
ecm: 30/30 curves on C154, B1=2k, B2=gmp-ecm default
ecm: 74/74 curves on C154, B1=11k, B2=gmp-ecm default
ecm: 214/214 curves on C154, B1=50k, B2=gmp-ecm default, ETA: 0 sec
pm1: starting B1 = 3750K, B2 = gmp-ecm default on C154
ecm: 430/430 curves on C154, B1=250k, B2=gmp-ecm default, ETA: 0 sec
pm1: starting B1 = 15M, B2 = gmp-ecm default on C154
ecm: 904/904 curves on C154, B1=1M, B2=gmp-ecm default, ETA: 0 sec
ecm: 2350/2350 curves on C154, B1=3M, B2=gmp-ecm default, ETA: 0 sec
ecm: 4480/4480 curves on C154, B1=11M, B2=gmp-ecm default, ETA: 0 sec
ecm: 2453/2453 curves on C154, B1=43M, B2=gmp-ecm default, ETA: 0 sec
nfs: commencing nfs on c154: 7481941016577287348099705569618881449446613674024887958060868042015962068162606112448282331657456711392343427317702862486293906907971066705762469196914429
nfs: searching for brent special forms...
nfs: searching for homogeneous cunningham special forms...
nfs: searching for XYYXF special forms...
nfs: couldn't find special form
nfs: commencing polynomial search over range: 8192 - 8442
nfs: commencing polynomial search over range: 8692 - 8942
nfs: commencing polynomial search over range: 8942 - 9192
nfs: commencing polynomial search over range: 8442 - 8692
nfs: commencing polynomial search over range: 9692 - 9942
nfs: commencing polynomial search over range: 9442 - 9692
nfs: commencing polynomial search over range: 10192 - 10442
nfs: commencing polynomial search over range: 9192 - 9442
nfs: commencing polynomial search over range: 10692 - 10942
nfs: commencing polynomial search over range: 11192 - 11442
nfs: commencing polynomial search over range: 11442 - 11692
nfs: commencing polynomial search over range: 11692 - 11942
nfs: commencing polynomial search over range: 11942 - 12192
nfs: commencing polynomial search over range: 9942 - 10192
nfs: commencing polynomial search over range: 10942 - 11192
nfs: commencing polynomial search over range: 10442 - 10692
deadline: 8640000 CPU-seconds per coefficient
deadline: 8640000 CPU-seconds per coefficient
deadline: 8640000 CPU-seconds per coefficient
deadline: 8640000 CPU-seconds per coefficient
deadline: 8640000 CPU-seconds per coefficient
deadline: 8640000 CPU-seconds per coefficient
deadline: 8640000 CPU-seconds per coefficient
deadline: 8640000 CPU-seconds per coefficient
deadline: 8640000 CPU-seconds per coefficient
deadline: 8640000 CPU-seconds per coefficient
deadline: 8640000 CPU-seconds per coefficient
deadline: 8640000 CPU-seconds per coefficient
deadline: 8640000 CPU-seconds per coefficient
deadline: 8640000 CPU-seconds per coefficient
coeff 11760 specialq 1 - 15246007 other 33328 - 79987
Segmentation fault (core dumped)
[/code]Thanks,

EdH 2020-08-13 14:48

I think I failed to mention the msieve edit in my first post and this might be what's happening here. If you haven't made the msieve edit, check out [URL="https://www.mersenneforum.org/showpost.php?p=540947&postcount=81"]post #81[/URL] and edit the appropriate file and then recompile both msieve and YAFU and see if that fixes it.

jasonp 2020-09-04 16:44

Sorry for the huge delay, fix has been committed to Msieve trunk.

EdH 2020-09-04 18:32

[QUOTE=jasonp;556030]Sorry for the huge delay, fix has been committed to Msieve trunk.[/QUOTE]
Thanks! I'll remove the edit references from the initial post sometime soon.

RedGolpe 2020-09-28 17:31

Hi all, Makefile currently contains the following code

[CODE]# modify the following line for your particular msieve installation
ifeq ($(KNC),1)
LIBS += -L../msieve/
else
ifeq ($(COMPILER),icc)
LIBS += -L../msieve/lib/linux
else
LIBS += -L../msieve/lib/linux/gcc73/
endif
endif
[/CODE]
in place of the one shown in the first post. I just changed the whole external ifeq bloc with
[CODE]
LIBS += -L../msieve
[/CODE]
and it seems to work but you might want to update the instructions.


All times are UTC. The time now is 05:53.

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.