mersenneforum.org  

Go Back   mersenneforum.org > Factoring Projects > Msieve

Reply
 
Thread Tools
Old 2009-03-03, 06:35   #1
mdettweiler
A Sunny Moo
 
mdettweiler's Avatar
 
Aug 2007
USA (GMT-5)

792 Posts
Default Postprocessing error: "too many merge attempts"

Just today I was using msieve to postprocess a GNFS job on a C101 which, by accident, had been very, very oversieved. During the filtering stage it kept giving me a "too many merge attempts" error; here's the sequence of events of how I encountered this error message and attempted to fix it:

-Sieving had collected almost 7 million relations, compared to about 5.5 million which was required for this job. I started msieve v1.34 (why I was using such an antiquated version is a somewhat long story) and it ran until it gave the "too many merge attempts" error after attempting to improve the matrix a number of times. The log file from this attempt is included in the attached archive as "msieve.log.134".

-I then tried re-running the job with msieve v1.40 beta. This also failed; by my admittedly quite limited understanding of how all this works, it appeared to do a little better in its attempts to improve the matrix, but still ultimately failed. The log file from this attempt is included as "msieve.log.140".

-Remembering a thread a while back that discussed problems that cropped up with humongously oversieved jobs in postprocessing, I tried whacking the # of relations down to about 6.5 million and ran it with v1.40. I couldn't tell whether this improved things much (though my later results indicated that this probably did help at least to some extent), and didn't save a log file from this attempt.

-I cut the # of relations down even more, to 5.75 million, and ran postprocessing with v1.40. This time it worked (somewhat fitfully, it appeared, though again my knowledge of how this stuff works is still somewhat limited and thus I could be dead wrong on this ) and produced factors. The log from this attempt is included as "msieve.log.140.final".

It appears that the problem lies in the sheer amount of oversieving that was done on this number. I'm pretty sure that msieve's behavior in circumstances like this has been reported at least once before, but I figured I'd report this occurrence in case it ends up being helpful to improve the program.

One other, somewhat unrelated bug(?) that I noticed during the final and successful attempt to run the postprocessing was that despite the fact that I ran msieve with the "-t 2" command line flag to run on both cores of my computer, it only ran single-threaded during the linear algebra stage. I'm not sure if this is a bug, or if this is somehow this was due to the earlier wonkiness due to all the extra relations.

Anyway, just figured I'd post the logs from this somewhat difficult job in case it proves enlightening to anyone.

Max
Attached Files
File Type: bz2 msieve.logs.tar.bz2 (8.4 KB, 66 views)
mdettweiler is offline   Reply With Quote
Old 2009-03-03, 07:22   #2
10metreh
 
10metreh's Avatar
 
Nov 2008

2·33·43 Posts
Default

Quote:
Originally Posted by mdettweiler View Post
Just today I was using msieve to postprocess a GNFS job on a C101 which, by accident, had been very, very oversieved. During the filtering stage it kept giving me a "too many merge attempts" error; here's the sequence of events of how I encountered this error message and attempted to fix it:

-Sieving had collected almost 7 million relations, compared to about 5.5 million which was required for this job. I started msieve v1.34 (why I was using such an antiquated version is a somewhat long story) and it ran until it gave the "too many merge attempts" error after attempting to improve the matrix a number of times. The log file from this attempt is included in the attached archive as "msieve.log.134".

-I then tried re-running the job with msieve v1.40 beta. This also failed; by my admittedly quite limited understanding of how all this works, it appeared to do a little better in its attempts to improve the matrix, but still ultimately failed. The log file from this attempt is included as "msieve.log.140".

-Remembering a thread a while back that discussed problems that cropped up with humongously oversieved jobs in postprocessing, I tried whacking the # of relations down to about 6.5 million and ran it with v1.40. I couldn't tell whether this improved things much (though my later results indicated that this probably did help at least to some extent), and didn't save a log file from this attempt.

-I cut the # of relations down even more, to 5.75 million, and ran postprocessing with v1.40. This time it worked (somewhat fitfully, it appeared, though again my knowledge of how this stuff works is still somewhat limited and thus I could be dead wrong on this ) and produced factors. The log from this attempt is included as "msieve.log.140.final".

It appears that the problem lies in the sheer amount of oversieving that was done on this number. I'm pretty sure that msieve's behavior in circumstances like this has been reported at least once before, but I figured I'd report this occurrence in case it ends up being helpful to improve the program.

One other, somewhat unrelated bug(?) that I noticed during the final and successful attempt to run the postprocessing was that despite the fact that I ran msieve with the "-t 2" command line flag to run on both cores of my computer, it only ran single-threaded during the linear algebra stage. I'm not sure if this is a bug, or if this is somehow this was due to the earlier wonkiness due to all the extra relations.

Anyway, just figured I'd post the logs from this somewhat difficult job in case it proves enlightening to anyone.

Max
www.mersenneforum.org/showthread.php?t=11271
www.mersenneforum.org/showthread.php?p=152180

Last fiddled with by 10metreh on 2009-03-03 at 07:29
10metreh is offline   Reply With Quote
Old 2009-03-03, 07:35   #3
Batalov
 
Batalov's Avatar
 
"Serge"
Mar 2008
Phi(4,2^7658614+1)/2

22×2,281 Posts
Default

Quote:
Originally Posted by mdettweiler View Post
One other, somewhat unrelated bug(?) that I noticed during the final and successful attempt to run the postprocessing was that despite the fact that I ran msieve with the "-t 2" command line flag to run on both cores of my computer, it only ran single-threaded during the linear algebra stage. I'm not sure if this is a bug, or if this is somehow this was due to the earlier wonkiness due to all the extra relations.
Well, you know, the help of more than 1 thread can turn into a disservice, that's why it is not a bug, but a feature. It is built-in to do so (under a certain size). It is not related.

You know, in short, BL scaling up is very sublinear, and the less the job, the less sense -t 2 makes. So it is forced to be -t 1 in the code under a certain size. Even for large (but not very large) jobs, on a more-than-4-cpu cluster, I've learned to live and let live and not use more than -t 4. (live and let live? Just watched Bond-21... Rubbish. )
Batalov is offline   Reply With Quote
Old 2009-03-03, 13:12   #4
jasonp
Tribal Bullet
 
jasonp's Avatar
 
Oct 2004

3·1,163 Posts
Default

The linear algebra uses one thread for any matrix whose size is smaller than 250k; multithreaded support adds a lot of overhead that would make small jobs take longer.
jasonp is offline   Reply With Quote
Old 2009-03-03, 16:53   #5
mdettweiler
A Sunny Moo
 
mdettweiler's Avatar
 
Aug 2007
USA (GMT-5)

792 Posts
Default

Thanks for the links--those do indeed seem to be the same problem I had and they proved to be quite enlightening.
Quote:
Originally Posted by Batalov View Post
Well, you know, the help of more than 1 thread can turn into a disservice, that's why it is not a bug, but a feature. It is built-in to do so (under a certain size). It is not related.

You know, in short, BL scaling up is very sublinear, and the less the job, the less sense -t 2 makes. So it is forced to be -t 1 in the code under a certain size. Even for large (but not very large) jobs, on a more-than-4-cpu cluster, I've learned to live and let live and not use more than -t 4. (live and let live? Just watched Bond-21... Rubbish. )
Quote:
Originally Posted by jasonp View Post
The linear algebra uses one thread for any matrix whose size is smaller than 250k; multithreaded support adds a lot of overhead that would make small jobs take longer.
Ah, I get it now. Thanks guys for explaining.
mdettweiler is offline   Reply With Quote
Reply

Thread Tools


Similar Threads
Thread Thread Starter Forum Replies Last Post
"Merge CPUs" removed them completly? KradziejSoku PrimeNet 6 2020-10-16 11:04
"lanczos error: only trivial dependencies found" with massive oversieving eigma Msieve 21 2015-05-28 03:27
Refurb. Mac emits "Bad CPU type in executable" error ewmayer Hardware 15 2014-08-11 22:03
Too many merge attempts etc. hhh Aliquot Sequences 28 2009-05-14 19:54
Would Minimizing "iterations between results file" may reveal "is not prime" earlier? nitai1999 Software 7 2004-08-26 18:12

All times are UTC. The time now is 12:03.

Fri Oct 23 12:03:04 UTC 2020 up 43 days, 9:14, 0 users, load averages: 1.33, 1.30, 1.32

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.