mersenneforum.org  

Go Back   mersenneforum.org > Factoring Projects > YAFU

Reply
 
Thread Tools
Old 2016-09-15, 14:38   #12
bsquared
 
bsquared's Avatar
 
"Ben"
Feb 2007

1101111110102 Posts
Default

Quote:
Originally Posted by amphoria View Post
You are correct as I have just realised that the version number is also printed on every line in factor.log.

For completeness I have also built yafu and msieve using VS2013. Yafu and msieve were built from the latest trunk code. This built Yafu 1.34.5 also. However the issue with poly select deadline still exists, so it might be something related to "small" composites.
The latest trunk code has not changed; I've been doing all new development in \branches\wip, which should build as version "1.35-beta". You can get the latest by clicking on the code tab from the souceforge yafu home screen, then navigate to branches -> wip and click on download snapshot. I'm pretty sure it will build with VS2013; let me know if not.

Thanks for the info you posted. All I can say so far is that the newest code built for linux, with the latest msieve code also built for linux, works with no issues. I'll do some investigating and find out what is broken... maybe it is a windows thing.

Last fiddled with by bsquared on 2016-09-15 at 14:42
bsquared is offline   Reply With Quote
Old 2016-09-15, 17:57   #13
bsquared
 
bsquared's Avatar
 
"Ben"
Feb 2007

2×1,789 Posts
Default

Whatever the problem is, it is isolated to version 1.34.5 (or perhaps earlier) on windows systems:

v1.34.5
Code:
09/15/16 11:18:01 v1.34.5 @ WINDOWS-SYSTEM, nfs: commencing nfs on c99: 741244879109376246366513985428555418845913568310626508732412421094649581664407020182598936240636611
09/15/16 11:18:01 v1.34.5 @ WINDOWS-SYSTEM, nfs: commencing poly selection with 4 threads
09/15/16 11:18:01 v1.34.5 @ WINDOWS-SYSTEM, nfs: setting deadline of 297 seconds
09/15/16 11:22:46 v1.34.5 @ WINDOWS-SYSTEM, nfs: completed 56 ranges of size 5 in 284.8455 seconds
09/15/16 11:22:46 v1.34.5 @ WINDOWS-SYSTEM, nfs: best poly = # norm 1.646415e-013 alpha -4.744248 e 1.356e-008 rroots 4
09/15/16 12:36:22 v1.34.5 @ LINUX-SYSTEM: commencing nfs on c99: 741244879109376246366513985428555418845913568310626508732412421094649581664407020182598936240636611
09/15/16 12:36:22 v1.34.5 @ LINUX-SYSTEM: commencing poly selection with 8 threads
09/15/16 12:36:22 v1.34.5 @ LINUX-SYSTEM: setting deadline of 148 seconds
09/15/16 12:38:53 v1.34.5 @ LINUX-SYSTEM: completed 9173 ranges of size 5 in 150.9921 seconds
09/15/16 12:38:53 v1.34.5 @ LINUX-SYSTEM: best poly = # norm 1.197408e-13 alpha -4.840726 e 1.182e-08 rroots 2
v1.35-beta
Code:
09/15/16 09:50:01 v1.35-beta @ WINDOWS-VERSION, nfs: commencing nfs on c99: 741244879109376246366513985428555418845913568310626508732412421094649581664407020182598936240636611
09/15/16 09:50:02 v1.35-beta @ WINDOWS-VERSION, nfs: commencing poly selection with 4 threads
09/15/16 09:50:02 v1.35-beta @ WINDOWS-VERSION, nfs: setting deadline of 297 seconds
09/15/16 09:55:00 v1.35-beta @ WINDOWS-VERSION, nfs: completed 1096 ranges of size 5 in 298.5198 seconds
09/15/16 09:55:00 v1.35-beta @ WINDOWS-VERSION, nfs: best poly = # norm 1.334633e-013 alpha -4.735009 e 1.235e-008 rroots 2
09/15/16 12:52:00 v1.35-beta @ LINUX-VERSION, nfs: commencing nfs on c99: 741244879109376246366513985428555418845913568310626508732412421094649581664407020182598936240636611
09/15/16 12:52:00 v1.35-beta @ LINUX-VERSION, nfs: commencing poly selection with 8 threads
09/15/16 12:52:00 v1.35-beta @ LINUX-VERSION, nfs: setting deadline of 148 seconds
09/15/16 12:54:30 v1.35-beta @ LINUX-VERSION, nfs: completed 11525 ranges of size 5 in 150.0306 seconds
09/15/16 12:54:30 v1.35-beta @ LINUX-VERSION, nfs: best poly = # norm 1.195023e-13 alpha -4.787281 e 1.173e-08 rroots 4
It looks like the problem is in whatever version of msieve was linked into the 1.34.5 executable because the problem doesn't exist in today's version on windows or linux. There are lots of benefits to building the new yafu version but to solve this problem what you really need to do is build the newest msieve first. You should be able to do that and then use either 1.34.5 or 1.35-beta.
bsquared is offline   Reply With Quote
Old 2016-09-15, 19:08   #14
amphoria
 
amphoria's Avatar
 
"Dave"
Sep 2005
UK

277610 Posts
Default

Quote:
Originally Posted by bsquared View Post
It looks like the problem is in whatever version of msieve was linked into the 1.34.5 executable because the problem doesn't exist in today's version on windows or linux. There are lots of benefits to building the new yafu version but to solve this problem what you really need to do is build the newest msieve first. You should be able to do that and then use either 1.34.5 or 1.35-beta.
Unfortunately if I link the latest version of msieve (trunk r993) against yafu 1.35-beta, msieve is crashing during the linear algebra phase. Strangely I was able to link the same version of msieve against yafu 1.34.5 without it crashing, but this does not fix the poly select deadline problem.

Solving the crash is beyond my capabilities. I thought that it might be related to the pthreads library. The latest compiled version I could find is from 2012.

Do you have a working version of yafu 1.35-beta for windows that you can post somewhere?
amphoria is offline   Reply With Quote
Old 2016-09-15, 19:11   #15
pinhodecarlos
 
pinhodecarlos's Avatar
 
"Carlos Pinho"
Oct 2011
Milton Keynes, UK

115538 Posts
Default

My msieve version, windows 64 bit.
Attached Files
File Type: zip msieve.zip (554.0 KB, 146 views)
pinhodecarlos is offline   Reply With Quote
Old 2016-09-15, 20:52   #16
bsquared
 
bsquared's Avatar
 
"Ben"
Feb 2007

2·1,789 Posts
Default

Quote:
Originally Posted by amphoria View Post
Unfortunately if I link the latest version of msieve (trunk r993) against yafu 1.35-beta, msieve is crashing during the linear algebra phase. Strangely I was able to link the same version of msieve against yafu 1.34.5 without it crashing, but this does not fix the poly select deadline problem.

Solving the crash is beyond my capabilities. I thought that it might be related to the pthreads library. The latest compiled version I could find is from 2012.

Do you have a working version of yafu 1.35-beta for windows that you can post somewhere?
You are right, it does crash in LA. I had never run a NFS job to completion on windows with the new version. Well... one more thing to work out.
bsquared is offline   Reply With Quote
Old 2016-09-16, 03:55   #17
EdH
 
EdH's Avatar
 
"Ed Hall"
Dec 2009
Adirondack Mtns

76378 Posts
Default

I'm having a problem that may be related, but not sure. I'm limited for time right now. I was going to wait until I could gather some more data, but since it might be relative, here's some of what I have:

I have a number that won't factor on four different linux machines using the newest versions of all the packages. The runs seem to drop out with no messages, all the way to the command prompt (edit - YAFU was started from the command prompt.):
Code:
12936 45949468771135 4360973942040003788613537
12936 8626660180883 4360973941819841271959991
12936 32826124443295 4360973940891776402571718
12936 58188657626405 4360973941635782096255512
save 1.835027e-14 -4.6541 3235478.33 4.963615e-09 rroots 2
save 1.734181e-14 -4.5970 3434628.62 4.825250e-09 rroots 2
hashtable: 1024 entries,  0.02 MB
elapsed time of 905.3081 seconds exceeds 803 second deadline; poly select done
nfs: commencing algebraic side lattice sieving over range: 1120000 - 1130000
nfs: commencing algebraic side lattice sieving over range: 1110000 - 1120000

...(lines removed for efficiency)

nfs: commencing algebraic side lattice sieving over range: 1480000 - 1490000
nfs: commencing algebraic side lattice sieving over range: 1470000 - 1480000
nfs: commencing msieve filtering
4678794529377765619886998533986913620849434520921582732844299916951588150985105814225896836243076543231
nfs: commencing algebraic side lattice sieving over range: 1500000 - 1510000
nfs: commencing algebraic side lattice sieving over range: 1490000 - 1500000
nfs: commencing msieve filtering
4678794529377765619886998533986913620849434520921582732844299916951588150985105814225896836243076543231
I have all the files that were created as well as a file of the YAFU output which I redirected. (Some of that file, including the end is shown above.) I can zip them up and put them on SendSpace if someone would like to see them, but it wouldn't be until tomorrow night.

BTW, I do have the factors. One of my machines found them with ECM:
Code:
prp45 = 508796826600370749062703008999220415547332823
prp58 = 9195801319438410752790284108566289031895884501156735887897

Last fiddled with by EdH on 2016-09-16 at 03:57 Reason: clarification
EdH is offline   Reply With Quote
Old 2016-09-16, 12:14   #18
amphoria
 
amphoria's Avatar
 
"Dave"
Sep 2005
UK

23·347 Posts
Default

Quote:
Originally Posted by bsquared View Post
You are right, it does crash in LA. I had never run a NFS job to completion on windows with the new version. Well... one more thing to work out.
If I have manged to debug it correctly, the crash is caused by a divide by zero error on line 526 of msieve\common\lancsoz\lancsoz_matmul0.c. Basically superblock_size is being set to 0 on line 496. obj->cache_size to is set to 6 on my PC (an i7-5960X) which seems very low. On the version of msieve linked to yafu 1.34.5 the processor cache size is 20480 kB according to nfs.log.

I am going to try building yafu 1.34.5 with the latest msieve again as it appeared to work last time. It it does, then the problem would appear to lie with yafu, otherwise it would appear to be a msieve problem.

If it works on linux, then may be msieve has a problem detecting the L3 cache size on windows.
amphoria is offline   Reply With Quote
Old 2016-09-16, 14:37   #19
bsquared
 
bsquared's Avatar
 
"Ben"
Feb 2007

2×1,789 Posts
Default

Quote:
Originally Posted by amphoria View Post
If I have manged to debug it correctly, the crash is caused by a divide by zero error on line 526 of msieve\common\lancsoz\lancsoz_matmul0.c. Basically superblock_size is being set to 0 on line 496. obj->cache_size to is set to 6 on my PC (an i7-5960X) which seems very low. On the version of msieve linked to yafu 1.34.5 the processor cache size is 20480 kB according to nfs.log.

I am going to try building yafu 1.34.5 with the latest msieve again as it appeared to work last time. It it does, then the problem would appear to lie with yafu, otherwise it would appear to be a msieve problem.

If it works on linux, then may be msieve has a problem detecting the L3 cache size on windows.
Thanks for looking into this. We'll see how your test goes, but I'm guessing that something in the interface changed in msieve so that memory allocations are getting corrupted between yafu/msieve. That happened once before and is a consequence of my shoddy half-way merge/link of the two programs. I will go and look at the revision log of msieve.
bsquared is offline   Reply With Quote
Old 2016-09-16, 15:43   #20
amphoria
 
amphoria's Avatar
 
"Dave"
Sep 2005
UK

23·347 Posts
Default

Quote:
Originally Posted by amphoria View Post
I am going to try building yafu 1.34.5 with the latest msieve again as it appeared to work last time. It it does, then the problem would appear to lie with yafu, otherwise it would appear to be a msieve problem.

If it works on linux, then may be msieve has a problem detecting the L3 cache size on windows.
It looks I did something wrong last time or, less likely, msieve is behaving differently now that I am building debug versions. If I link msieve r993 against yafu 1.34.5, then I still get the crash in LA. I didn't run it inside the VS debugger but nfs.log shows a superblock size of 0. Also the poly select deadline works correctly, so that must have been due to an old version of msieve linked to the windows binary on SourceForge.

This (the LA problem) is starting to look like a problem with msieve on windows detecting the L3 cache size.
amphoria is offline   Reply With Quote
Old 2016-09-16, 17:21   #21
amphoria
 
amphoria's Avatar
 
"Dave"
Sep 2005
UK

AD816 Posts
Default

Quote:
Originally Posted by amphoria View Post
This (the LA problem) is starting to look like a problem with msieve on windows detecting the L3 cache size.
OK the problem is with yafu calling the msieve library. I tested this by running poly select and sieving phases only from yafu, and then using the msieve binary to run filter, LA and sq root phases. Using msieve.exe the L3 cache size was correctly identified as 20480 kB and the superblock size calculated to be 1966080, and the LA ran successfully.
amphoria is offline   Reply With Quote
Old 2016-09-16, 17:37   #22
bsquared
 
bsquared's Avatar
 
"Ben"
Feb 2007

67728 Posts
Default

Quote:
Originally Posted by amphoria View Post
OK the problem is with yafu calling the msieve library. I tested this by running poly select and sieving phases only from yafu, and then using the msieve binary to run filter, LA and sq root phases. Using msieve.exe the L3 cache size was correctly identified as 20480 kB and the superblock size calculated to be 1966080, and the LA ran successfully.
My suspicions confirmed. Thank you for your testing - I'll get it fixed.
bsquared is offline   Reply With Quote
Reply

Thread Tools


Similar Threads
Thread Thread Starter Forum Replies Last Post
Running YAFU via Aliqueit doesn't find yafu.ini EdH YAFU 8 2018-03-14 17:22
A Desperate appeal! (by Richard K. Guy)... deadline is September 30, 2016 philmoore Factoring 102 2016-10-01 12:16
msieve poly select: choosing Stage1norm VBCurtis Msieve 0 2016-04-11 21:33
Starting NFS skipping poly select jux YAFU 5 2016-01-02 01:01
Time it takes to select polynomials for 154 digits John5788 Factoring 23 2008-08-27 07:54

All times are UTC. The time now is 00:16.


Mon Oct 18 00:16:04 UTC 2021 up 86 days, 18:45, 0 users, load averages: 1.53, 1.39, 1.33

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.