Too many merge attempts etc.

[schickel]

Originally Posted by hhh

Dear friends,

the "too many merge attempts"-error showed up at some c104 of mine. I tried to fiddle aroud a bit, with the sole success to replace it by some "the matrix must have more coloums than rows" (or vice versa) error. Here are my options, I guess:

- I upload thezipped folder to some volunteer who does the postprocessing for me
- You help me to get out of this mess
- I restart sieving, with perhaps slightly different parameters
- Somebody else does this number from scratch.

This is the order of my preference, also. You decide, though.

Cheers, H.

H, what version of msieve were you running? I keep forgetting to upgrade this one, so when I ran this, it was with 1.39 and it finished on the first try, so I don't know what zone you were really in:

linear algebra completed 222519 of 223174 dimensions (99.7%, ETA 0h 0m)
lanczos halted after 3528 iterations (dim = 222925)
recovered 32 nontrivial dependencies

commencing square root phase
reading relations for dependency 1
read 111749 cycles
cycles contain 455893 unique relations
read 455893 relations
multiplying 366356 relations
multiply complete, coefficients have about 14.51 million bits
initial square root is modulo 215881931
prp52 factor: 7570380941541769441561745312556684037938683763298177
prp52 factor: 8145994460731835857997597172380480561830720426463929
elapsed time 00:29:25

Nice split there!!!

[hhh]

Msieve 1.41. So, the problem was not oversieving, but the brilliance. Perhaps we should tell the case to the maker of Msieve, for that he knows that some functionality is lost in version 1.41.

Anyway, cheers, H.

[10metreh]

Originally Posted by hhh

Msieve 1.41. So, the problem was not oversieving, but the brilliance. Perhaps we should tell the case to the maker of Msieve, for that he knows that some functionality is lost in version 1.41.

Anyway, cheers, H.

When did nice splits make factorizations fail?

[schickel]

Originally Posted by batalov

2) (rarely visited; you are here!) almost enough relations; -nc1 makes cycles, -nc2 builds a matrix, cleans it up and then the matrix is not useable => Sieve some more;

Looking at the relation count, I think this is where you were, h, one more sieve block might have done it......looking back at similar sized jobs, mine needed slightly more relations than you had.

Originally Posted by 10metreh

When did nice splits make factorizations fail?

Probably not fail, maybe just need a few more relations than usual. It'd be interesting to see if other folks have had any problems with 50/50 splits on their jobs.

Maybe the solution is to keep 1.39 around as a fallback if there are similar problems in the future.

[hhh]

Originally Posted by 10metreh

When did nice splits make factorizations fail?

You,　here, post number 4. Actually, you didn't, but I understood it like that, my bad.

I was inclined to such a suggestion as you had some similar problem here with some p47*p49. I can be completely mistaken.

Looking at the relation count, I think this is where you were, h, one more sieve block might have done it......looking back at similar sized jobs, mine needed slightly more relations than you had.

I would have done that, hadn't msieve (or the perl script, I don't know) given me the error message.

Anyway, we have learned something (at least I have), that's something,too. Cheers, H.

[hhh]

OK, here we go again.

Thu May 07 13:29:45 2009 distribution of cycle lengths:
Thu May 07 13:29:45 2009 1 relations: 32601
Thu May 07 13:29:45 2009 2 relations: 161182
Thu May 07 13:29:45 2009 3 relations: 0
Thu May 07 13:29:45 2009 4 relations: 0
Thu May 07 13:29:45 2009 5 relations: 0
Thu May 07 13:29:45 2009 6 relations: 0
Thu May 07 13:29:45 2009 7 relations: 0
Thu May 07 13:29:45 2009 8 relations: 0
Thu May 07 13:29:45 2009 9 relations: 0
Thu May 07 13:29:45 2009 10+ relations: 0
Thu May 07 13:29:45 2009 heaviest cycle: 2 relations
Thu May 07 13:29:45 2009 matrix not dense enough, retrying

error: too many merge attempts
Return value 65280. Terminating...
WARNING: gnfs failed to find a factor. This really shouldn't happen.
etc.

The last two lines were visible on the screen only, but I copied them into the log. When I restarted aliqueit, it sieved again a little bit, same story, and doesn't take up sieving again, now. Can you please help me to finish this job by hand?

[henryzz]

have you tried again?

[jasonp]

This job is very heavily oversieved; try it with ~2/3 the total number of relations (it was not oversieved enough to trigger the new code in v1.41 for dealing with very heavy oversieving).

[schickel]

Originally Posted by jasonp

This job is very heavily oversieved; try it with ~2/3 the total number of relations (it was not oversieved enough to trigger the new code in v1.41 for dealing with very heavy oversieving).

I wonder if he's in the same position as last time....just a tiny bit under the required number of realtions. My older c112 jobs, all done completely with GGNFS and the factlat.pl script, required between 7 and 8 million relations. From hhh's logfile, he's at this position:

found 1261318 hash collisions in 8230451 relations
commencing duplicate removal, pass 2
found 2115181 duplicates and 6115270 unique relations

Which says that he probably needs at least one more block of sieving, if not a couple more. Look at that duplicate rate.....nearly 25%

hhh, I can take another stab at it if you like, or you can try one more block of sieving and see what happens......let me know and we can setup an FTP transfer again.

[hhh]

I have got the impression that when I restart aliqueit, it keeps resieving the same spot. How can I manually sieve a chunk and manually start the postprocessing? That would be of much help. Thanks for all your concern, H.

[schickel]

Originally Posted by hhh

I have got the impression that when I restart aliqueit, it keeps resieving the same spot. How can I manually sieve a chunk and manually start the postprocessing? That would be of much help. Thanks for all your concern, H.

Maybe someone else can chime in, since I only run aliqueit on sequences until they hit 100-digits; but do you only run NFS jobs under aliqueit? My solution would be to use one of the fact*.pl scripts and adjust the .job file to move the next chunks up above where it's already been sieved.