mersenneforum.org  

Go Back   mersenneforum.org > Factoring Projects > Msieve

Reply
 
Thread Tools
Old 2013-07-14, 05:47   #166
wombatman
I moo ablest echo power!
 
wombatman's Avatar
 
May 2013

5×349 Posts
Default

Quick question--if I compile the latest revision of msieve (SVN 916), will that include the updated linear algebra code that significantly sped up that portion of the work?

On a related note, I compiled SVN 916 for my AMD64 with and without CUDA and ECM. I haven't fully tested them--only used the CUDA/ECM exe to perform stage 1 & 2 polynomial searching.

Edit: Just wanted to note that this was compiled on WinXP 32-bit with -march=native.

Last fiddled with by wombatman on 2013-07-14 at 06:07
wombatman is offline   Reply With Quote
Old 2013-07-14, 10:03   #167
Brian Gladman
 
Brian Gladman's Avatar
 
May 2008
Worcester, United Kingdom

52610 Posts
Default

Quote:
Originally Posted by WraithX View Post
At this point, I'm not sure if the Windows builds can handle large jobs. I'm still trouble shooting my MinGW64 build. It turns out that it works just fine on a GNFS139 I had sitting on my hard drive. That only had a 3.02GB dat file. However, neither my MinGW64 nor Brian's Visual Studio build have worked on the L1911 24GB dat file. I think they read in all of the relations just fine, there just seem to be strange problems after that.
I compiled without large address awareness, which I imagine might cause problems with large data sets. I have sent Jeff a version compiled with large address awareness, which might be worth trying. I have also attached it here (hoping the mods won't mind).
Attached Files
File Type: zip msieve.zip (619.2 KB, 70 views)
Brian Gladman is offline   Reply With Quote
Old 2013-07-14, 11:05   #168
Jeff Gilchrist
 
Jeff Gilchrist's Avatar
 
Jun 2003
Ottawa, Canada

3×17×23 Posts
Default

The version on the website how now been updated (same filename but newer binary inside)
Jeff Gilchrist is offline   Reply With Quote
Old 2013-07-14, 23:38   #169
WraithX
 
WraithX's Avatar
 
Mar 2006

11×43 Posts
Default

Quote:
Originally Posted by Brian Gladman View Post
I compiled without large address awareness, which I imagine might cause problems with large data sets. I have sent Jeff a version compiled with large address awareness, which might be worth trying. I have also attached it here (hoping the mods won't mind).
At this point I've tried your Svn915 and your Svn915-Large-Address-Aware(LAA) and they have both crashed in the same spot on L1911, which is a GNFS167 with a 24GB dat file. Actually, I have also tried both of your binaries on a GNFS139 which had a 3GB dat file. All four tests crashed in the same spot, right after "Reading all ideals from disk". I dont' think LAA needs to be used in a 64-bit binary. That should only be needed with 32-bit binaries. One thing I'm wondering about, why do all of your compiles say i7? Could you try to compile a Core-2 binary and let me test that out? I don't know if there might be some instruction getting compiled in that my processors don't like, or if your binaries aren't working for me for some other reason. Speaking of which, can anyone else test this out to see if it will get to the linear algebra phase, or if it crashes before then?

Here is my log file from your SVN915 (both regular and LAA produced the same log file) on the GNFS139, in case this can help with any troubleshooting steps we might need to take. And, to hopefully make clear what I am thinking, the crashes I'm seeing from Brian's Visual Studio Win64 binaries are different from the crashes I'm seeing with my MinGW64 Win64 binaries.
Attached Files
File Type: txt msieve-c139.log.txt (4.3 KB, 68 views)
WraithX is offline   Reply With Quote
Old 2013-07-15, 07:37   #170
Brian Gladman
 
Brian Gladman's Avatar
 
May 2008
Worcester, United Kingdom

2×263 Posts
Default

Quote:
Originally Posted by WraithX View Post
At this point I've tried your Svn915 and your Svn915-Large-Address-Aware(LAA) and they have both crashed in the same spot on L1911, which is a GNFS167 with a 24GB dat file. Actually, I have also tried both of your binaries on a GNFS139 which had a 3GB dat file. All four tests crashed in the same spot, right after "Reading all ideals from disk". I dont' think LAA needs to be used in a 64-bit binary. That should only be needed with 32-bit binaries. One thing I'm wondering about, why do all of your compiles say i7? Could you try to compile a Core-2 binary and let me test that out? I don't know if there might be some instruction getting compiled in that my processors don't like, or if your binaries aren't working for me for some other reason. Speaking of which, can anyone else test this out to see if it will get to the linear algebra phase, or if it crashes before then?

Here is my log file from your SVN915 (both regular and LAA produced the same log file) on the GNFS139, in case this can help with any troubleshooting steps we might need to take. And, to hopefully make clear what I am thinking, the crashes I'm seeing from Brian's Visual Studio Win64 binaries are different from the crashes I'm seeing with my MinGW64 Win64 binaries.
I agree that Large address aware should not matter for x64 but I wondered if there was some idiosyncracy that made it necessary. It would be useful to know the size of the smallest example that produces a crash and what the size of the data is for this example when compressed. I am not able to cope with a 24 GB download but I might be able to debug a much smaller example if one exists that will make it reasonable to download the data (~1 GB max).

Here is msieve compiled for Core2:
Attached Files
File Type: zip msieve.core2.zip (619.0 KB, 79 views)

Last fiddled with by Brian Gladman on 2013-07-15 at 07:45 Reason: more information
Brian Gladman is offline   Reply With Quote
Old 2013-07-16, 00:31   #171
WraithX
 
WraithX's Avatar
 
Mar 2006

1D916 Posts
Default

Quote:
Originally Posted by Brian Gladman View Post
It would be useful to know the size of the smallest example that produces a crash and what the size of the data is for this example when compressed. I am not able to cope with a 24 GB download but I might be able to debug a much smaller example if one exists that will make it reasonable to download the data (~1 GB max).

Here is msieve compiled for Core2:
I've tried the Core2 compile and it too crashed on my C139 in the same spot as the earlier binaries. I then created a test GNFS number that is a C120 = P60*P60. I ran it through factmsieve.py with ggnfs and an old version of msieve. The gathering of relations and the msieve post-processing were all finished in about 5 hours. The dat file is 1.12GB and I was able to .tar.bz2 the .dat, the .ini, and the .fb file down to 465MB. When I ran this number through your SVN915 i7 and Core2 binaries I get a crash 2 minutes after starting the command "msieve -v -nc" in the same place as my previous crashes, ie right after "reading all ideals from disk". Brian, I can upload this test case to my web site, or I can give you access to my ftp server. PM me with how you would like to get this test case and I can PM you back the details of how/where to get it. Or, if you'd like a smaller test case I can probably make one of those too.
WraithX is offline   Reply With Quote
Old 2013-07-17, 07:50   #172
Jarod
AKA Speedy51
 
Jarod's Avatar
 
Oct 2012
New Zealand

21910 Posts
Default I can replicate the issue

Quote:
Originally Posted by WraithX View Post
I downloaded the svn915 from Jeff's site a little while ago. Unfortunately, even this version of msieve crashed on me while working on L1911. This time it crashed on me in a slightly different place. I'm attaching the log to see if anyone can see what might have been the problem this time.
I ran SVN 915 and I can confirm using a different number the one that is in the I seven test zip. It crashed in exactly the same place after about 5 minutes of running. I have attached my log
Attached Files
File Type: txt msieve.txt (1.7 KB, 73 views)
Jarod is offline   Reply With Quote
Old 2013-07-17, 12:35   #173
Brian Gladman
 
Brian Gladman's Avatar
 
May 2008
Worcester, United Kingdom

2·263 Posts
Default

Quote:
Originally Posted by Speedy51 View Post
I ran SVN 915 and I can confirm using a different number the one that is in the I seven test zip. It crashed in exactly the same place after about 5 minutes of running. I have attached my log
What are the inputs for this example and where are they? Are they small enough to contemplate a download?
Brian Gladman is offline   Reply With Quote
Old 2013-07-17, 22:03   #174
Brian Gladman
 
Brian Gladman's Avatar
 
May 2008
Worcester, United Kingdom

2·263 Posts
Default

Quote:
Originally Posted by WraithX View Post
I've tried the Core2 compile and it too crashed on my C139 in the same spot as the earlier binaries. I then created a test GNFS number that is a C120 = P60*P60. I ran it through factmsieve.py with ggnfs and an old version of msieve. The gathering of relations and the msieve post-processing were all finished in about 5 hours. The dat file is 1.12GB and I was able to .tar.bz2 the .dat, the .ini, and the .fb file down to 465MB. When I ran this number through your SVN915 i7 and Core2 binaries I get a crash 2 minutes after starting the command "msieve -v -nc" in the same place as my previous crashes, ie right after "reading all ideals from disk". Brian, I can upload this test case to my web site, or I can give you access to my ftp server. PM me with how you would like to get this test case and I can PM you back the details of how/where to get it. Or, if you'd like a smaller test case I can probably make one of those too.
Thanks to David's effort to provide me with a Windows test case that leads to a crash, I have found that the exception occurs at line 55 in filter.c because filter->lp_file_size is zero and no memory is allocated.

At line 159 of singleton.c the savefile msieve.dat.lp has been written and closed and the call "filter->lp_file_size = get_file_size(buf);" is used to get its size.

BUT get_file_size() doesn't find the file with this name (nor with ".gz" added) so zero length is returned. I don't yet know why as the file is present.

Last fiddled with by Brian Gladman on 2013-07-17 at 22:31 Reason: further information
Brian Gladman is offline   Reply With Quote
Old 2013-07-17, 22:57   #175
wombatman
I moo ablest echo power!
 
wombatman's Avatar
 
May 2013

110110100012 Posts
Default

I have a compiled version of SVN916 for Core-i7 64bit with and without both CUDA and ECM if anybody is interested. I tested with both polynomial selection and relation sieving briefly and didn't get any crashes. If you want one of them, just PM and I'll be happy to email it to you.

Ben
wombatman is offline   Reply With Quote
Old 2013-07-17, 23:14   #176
Brian Gladman
 
Brian Gladman's Avatar
 
May 2008
Worcester, United Kingdom

2·263 Posts
Default

Quote:
Originally Posted by Brian Gladman View Post
Thanks to David's effort to provide me with a Windows test case that leads to a crash, I have found that the exception occurs at line 55 in filter.c because filter->lp_file_size is zero and no memory is allocated.

At line 159 of singleton.c the savefile msieve.dat.lp has been written and closed and the call "filter->lp_file_size = get_file_size(buf);" is used to get its size.

BUT get_file_size() doesn't find the file with this name (nor with ".gz" added) so zero length is returned. I don't yet know why as the file is present.
I have found the issue - I was compiling for a UNICODE character set so the ASCII file names were not being recognised. I will shortly pass a corrected version to Jeff.

Thanks to WraithX for his help in finding this. I have added the x64 i7 version below as a ZIP file.

Brian
Attached Files
File Type: zip msieve.i7.zip (619.0 KB, 71 views)

Last fiddled with by Brian Gladman on 2013-07-17 at 23:28
Brian Gladman is offline   Reply With Quote
Reply

Thread Tools


Similar Threads
Thread Thread Starter Forum Replies Last Post
Msieve 1.53 feedback xilman Msieve 149 2018-11-12 06:37
Msieve 1.50 feedback firejuggler Msieve 99 2013-02-17 11:53
Msieve 1.43 feedback Jeff Gilchrist Msieve 47 2009-11-24 15:53
Msieve 1.42 feedback Andi47 Msieve 167 2009-10-18 19:37
Msieve 1.41 Feedback Batalov Msieve 130 2009-06-09 16:01

All times are UTC. The time now is 03:13.

Sun May 16 03:13:41 UTC 2021 up 37 days, 21:54, 0 users, load averages: 2.24, 2.56, 2.77

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.