YAFU Version 2.10

[WraithX]

Originally Posted by bsquared

An unfortunate side effect of this is that when I add an option and update the repository yafu.ini, then a user's yafu.ini file (maybe full of custom things) gets overwritten when they update versions. Or is there a merge feature when fetching/pulling from github? I'm still a relative noob with git/github.

I'm not sure how others download and build their software, but when I have downloaded a git repository I do not edit any files in that directory. I copy the folder to a new directory and build there. Then I move that executable and ini file to other folders where I work on factoring projects. At that point I could delete the build folder, or save it for a backup copy of the exe. Running git pull in the original directory will update all the files. I can then copy that to a new directory and build the latest version, etc.

One nice thing about git though is that if you have changed a file, it will see there is a "conflict" with the master repository and it will not go through with the git pull. At that point you can do a git stash, which saves your changes to a local git stash, and then returns the directory to a clean state from the last git pull. Once the directory is in a clean state you can successfully run git pull. From here you have several git stash options:

git status
git stash list     (show list of stashed changes)
git stash          (save current changes to a stash and revert back to a clean/unmodified state, can git pull after this)
git stash show -p  (show diff between stash and branch)
git stash apply    (apply the top most stash to the current branch)
git stash drop     (drop the top most stash from the git stash stack *** WARNING: Make sure to "apply" first!  Verify with "status" *** )

So, if you've made a change to the ini file, a possible sequence of steps would be:

git stash        (save local changes to yafu.ini to a stash)
git pull         (get latest code changes)
git stash apply  (apply most recent stash, ie local edits to yafu.ini)
git stash drop   (drop most recent stash ***only drop if the apply was successful***)

Alternatively, you can remove yafu.ini from your repository and add a new file called yafu.ini.example. That way you can update the example file all you want and any user created yafu.ini file will be ignored by git since it isn't in the master branch.

[WraithX]

I have a question about yafu 2.10. I'm factoring a bunch of c70-c100 numbers and I have given the command line parameter -threads 1, and when yafu first starts up, it looks like it is fully using between 4-8 cores (I'm guessing whatever is free when it starts up). ie, when yafu starts it is bringing my cpu to 100%. Then after maybe 4-5 seconds, of factoring a single number, yafu will then settle down to only using a single core until it has finished factoring that number. Is it expected for yafu to use more than 1 core (thread?) when starting factoring?

A little more info if it helps: there shouldn't be a yafu.ini involved. Here is the command I am running:

./yafu-x64.exe -threads 1 -xover 105 -snfs_xover 105 -inmem 105 factor(<n>)

where n is an actual number I'm factoring (ie, not calling in batch mode). Am I missing an option to have it only use one thread when starting?

[kruoli]

It might be that what you see is overhead from spawning tons of ECM processes. Or maybe your GMP-ECM has been compiled with OpenMP enabled (only matters for P±1)?

[bsquared]

Nothing is threaded until you get to ECM and as far as I'm aware everything that is threaded is controlled by the threads option. The first thing it does is read/parse/set options and those are placed into the objects that are used everywhere.

After digging a bit I am also seeing a brief spike up to 100% at the end of p-1, so maybe during stage 2? I used this line when investigating:
./yafu-x64.exe -threads 1 -xover 105 -snfs_xover 105 -inmem 105 "pm1(rsa(330))" -B1pm1 100000000

Let me know if that seems like what you are seeing.

P-1 only uses the internally-linked gmp-ecm library, not an external ecm executable, so if you are using the yafu-x64.exe I provided then I guess I must have built it with openMP enabled? Apologies! I didn't mean to do that. A new version is coming soon; I'll add this to the fix list.

[Tyler Busby]

I'm getting an error when factoring (pardon me, I forget the compact expression, but it's of the form (2^k-k+1)*2^k+1))

yafu-x64.exe "factor(306180206916083902309240650087602475282639486413866622577088471913520022894784390350900738050555132295896611834140573938007698807080595783547161633771067442891542883495201433996626311245589014034513921)"

It's giving an "integrator failed" message and quitting:

>> fac: factoring 306180206916083902309240650087602475282639486413866622577088471913520022894784390350900738050555132295896611834140573938007698807080595783547161633771067442891542883495201433996626311245589014034513921
fac: using pretesting plan: normal
fac: using specified qs/gnfs crossover of 100 digits
fac: using specified qs/snfs crossover of 75 digits
div: primes less than 10000
fmt: 1000000 iterations
rho: x^2 + 3, starting 1000 iterations on C201
rho: x^2 + 2, starting 1000 iterations on C201
rho: x^2 + 1, starting 1000 iterations on C201
nfs: searching for brent special forms...
nfs: input divides 17498005798264095394980017816940970922825355447145699491406164851279623993595007385788105416184430426^2 - 27555
nfs: snfs form detection took 0.007000 seconds
fac: generating an SNFS polynomial to assess ECM effort
nfs: input divides 17498005798264095394980017816940970922825355447145699491406164851279623993595007385788105416184430426^2 - 27555
nfs: snfs form detection took 0.008000 seconds
integrator failed 7.290162e-51 4.194828e-57

I think this might also be responsible for cases in EdH's factordb collab where it seemingly cannot factor a number (no factor.log) and crashes the program. fwiw this seems to be happening on 2.9 and 2.10.

No hits of the text "integrator" in the yafu github makes me think it's throwing from a different library. And the fact that I can't even run ecm-py against it makes me think it's something to do with my ecm library. (ecm-py instantly completes whatever I ask of it without having found any factors). Although all references to "integrator failed" in the forum seem to be related to msieve.

[bsquared]

Originally Posted by Tyler Busby

I'm getting an error when factoring (pardon me, I forget the compact expression, but it's of the form (2^k-k+1)*2^k+1))

yafu-x64.exe "factor(306180206916083902309240650087602475282639486413866622577088471913520022894784390350900738050555132295896611834140573938007698807080595783547161633771067442891542883495201433996626311245589014034513921)"

It's giving an "integrator failed" message and quitting:

>> fac: factoring 306180206916083902309240650087602475282639486413866622577088471913520022894784390350900738050555132295896611834140573938007698807080595783547161633771067442891542883495201433996626311245589014034513921
fac: using pretesting plan: normal
fac: using specified qs/gnfs crossover of 100 digits
fac: using specified qs/snfs crossover of 75 digits
div: primes less than 10000
fmt: 1000000 iterations
rho: x^2 + 3, starting 1000 iterations on C201
rho: x^2 + 2, starting 1000 iterations on C201
rho: x^2 + 1, starting 1000 iterations on C201
nfs: searching for brent special forms...
nfs: input divides 17498005798264095394980017816940970922825355447145699491406164851279623993595007385788105416184430426^2 - 27555
nfs: snfs form detection took 0.007000 seconds
fac: generating an SNFS polynomial to assess ECM effort
nfs: input divides 17498005798264095394980017816940970922825355447145699491406164851279623993595007385788105416184430426^2 - 27555
nfs: snfs form detection took 0.008000 seconds
integrator failed 7.290162e-51 4.194828e-57

I think this might also be responsible for cases in EdH's factordb collab where it seemingly cannot factor a number (no factor.log) and crashes the program. fwiw this seems to be happening on 2.9 and 2.10.

No hits of the text "integrator" in the yafu github makes me think it's throwing from a different library. And the fact that I can't even run ecm-py against it makes me think it's something to do with my ecm library. (ecm-py instantly completes whatever I ask of it without having found any factors). Although all references to "integrator failed" in the forum seem to be related to msieve.

Yes, it's coming from msieve. It's because the snfs polynomial is ridiculous for this input although technically still of "special" form. If this happens use gnfs (-gnfs option).

[Tyler Busby]

Originally Posted by bsquared

Yes, it's coming from msieve. It's because the snfs polynomial is ridiculous for this input although technically still of "special" form. If this happens use gnfs (-gnfs option).

Thanks! That fixed it. The problem with ecm.py was because I had mixed up the order of the B1 and thread count arguments. I guess barring some sanity check from YAFU, the collab can just expect this error to happen sometime and grab a different composite.

[WraithX]

Originally Posted by bsquared

Nothing is threaded until you get to ECM and as far as I'm aware everything that is threaded is controlled by the threads option. The first thing it does is read/parse/set options and those are placed into the objects that are used everywhere.

After digging a bit I am also seeing a brief spike up to 100% at the end of p-1, so maybe during stage 2? I used this line when investigating:
./yafu-x64.exe -threads 1 -xover 105 -snfs_xover 105 -inmem 105 "pm1(rsa(330))" -B1pm1 100000000

Let me know if that seems like what you are seeing.

P-1 only uses the internally-linked gmp-ecm library, not an external ecm executable, so if you are using the yafu-x64.exe I provided then I guess I must have built it with openMP enabled? Apologies! I didn't mean to do that. A new version is coming soon; I'll add this to the fix list.

That might be it. When the pm1 started it was single threaded, then after a while it took as many cores as it could.

I was using the yafu exe from your github. I didn't even know it had OpenMP enabled. It wasn't a problem, but since I'm factoring so many numbers I did notice some slowdowns on my pc. Just wanted to let you know.

BTW, can I ask for another feature? :)
I know the yafu header is already pretty big. So, could you include a command to list all compile options, whether they are on or off?
Something like:
show_options or build_info
That will list compiled info and maybe even the header info again, like:

NFS: on
USE_SSE41: on
USE_AVX2: off
USE_BMI2: off
SKYLAKEX: off
ICELAKE: off
OpenMP: off (if on, maybe what version its using?)
<etc for other options I probably don't know about>

YAFU Version 2.10
Built with Microsoft Visual Studio 1931
Using GMP-ECM 7.0.4, Powered by MPIR 3.0.0
Detected Intel(R) Xeon(R) CPU E5-2687W v4 @ 3.00GHz
Detected L1 = 32768 bytes, L2 = 31457280 bytes, CL = 64 bytes
Using 1 random witness for Rabin-Miller PRP checks
Cached 664579 primes; max prime is 9999991
Parsed yafu.ini from <path>

Runtime options:
Threads: 1
Verbose Mode: 0
Seed: 42
LogFile: log.txt
GNFS/SIQS Crossover: 100
SNFS/SIQS Crossover: 75
Plan: normal
Inmem: 70

Oh, and maybe include some of the runtime options too so we can see what those are set to? Maybe that should be under a different command?

Thanks for updating yafu and considering all these requests!

[James Heinrich]

Originally Posted by WraithX

I know the yafu header is already pretty big. So, could you include a command to list all compile options, whether they are on or off?

I second that (or first-it, actually since I asked first ).

[bur]

When trying to do NFS using CADO I get this error:

ValueError: Given working directory must be an absolute path: cadoWorkdir

I see that cado-nfs.py is called with -w cadoWorkdir. I assume I have to assign a value to cadoWorkdir, but there's no such option in yafu.ini and simply adding it to yafu.ini results in invalid option cadoWorkdir.

[storm5510]

Originally Posted by James Heinrich

No, it's the Windows build. Note however that it's a link to a webpage from which you can download it, do not save-link-as directly -- open the link in your browser then click the Download button.
Or download it from https://download.mersenne.ca/YAFU if you prefer.

My browser, M$ Edge, refuses to download a binary file directly. All the .exe files need to be placed in archives of some type.