msieve help

[em99010pepe]

I'm using msieve with the following flags: -nc -t 4 -v
After 34 mins the client shutdowns without starting the linear algebra step. What am I doing wrong?
Thanks in advance,

Carlos

[fivemack]

How big a number are you working on, how many relations do you have, which OS are you using, and how much memory does the computer have?

[Andi47]

Does one of the last lines of your output (or the msieve.log file) say something like "filtering wants xy more relations"? In this case, you don't have enough relations yet and should do some more sieving.

[em99010pepe]

I am helping frmky. The number is 12,233-. I have a quad-core at 2.9 GHz running Vista 64-bit with 6 GB of memory and 16 GB of virtual memory.

Here's the latest log file.

http://em99010pepe.googlepages.com/msieve.log

I restarted a third run, it stopped again after 3 hours.

Thanks.

Carlos

[Andi47]

Originally Posted by em99010pepe

I am helping frmky. The number is 12,233-. I have a quad-core at 2.9 GHz running Vista 64-bit with 6 GB of memory and 16 GB of virtual memory.

Here's the latest log file.

http://em99010pepe.googlepages.com/msieve.log

I restarted a third run, it stopped again after 3 hours.

Thanks.

Carlos

Fri Sep 25 08:38:21 2009  reduce to 0 relations and 0 ideals in 4 passes
Fri Sep 25 08:38:21 2009  max relations containing the same ideal: 0
Fri Sep 25 08:38:21 2009  filtering wants 1000000 more relations
Fri Sep 25 08:38:21 2009  elapsed time 00:34:53

The line "filtering wants 1000000 more relations" at 8:28 am indicates that you had not enough relations to form a matrix at that time (which indicates that you needed more sieving).

"reduce to 0 relations and 0 ideals in 4 passes": when msieve attempts to reduce to one- or two-digit numbers of relations and ideals, that (usually, but not always?) indicates, that you are still relatively far from forming a matrix.

later...

Fri Sep 25 15:08:57 2009  matrix must have more columns than rows

Now it seems that you have barely enough relations to form a matrix, but not yet enough to finish the job. But you are pretty close.... Just do a little bit more sieving and then try again; 1 million more relations should do the trick.

[em99010pepe]

Between those runs I didn't do nothing. I only restarted msieve.

[Andi47]

Originally Posted by em99010pepe

Between those runs I didn't do nothing. I only restarted msieve.

Strange. If I read correctly, msieve started with 89M relations for the first run and 90M relations for the second one. Are you sure you didn't add a forgotten relations file to the msieve.dat file?

[em99010pepe]

Originally Posted by Andi47

Strange. If I read correctly, msieve started with 89M relations for the first run and 90M relations for the second one. Are you sure you didn't add a forgotten relations file to the msieve.dat file?

Nope, I am ignorant on the matter. Your explanation is Chinese for me!
One thing I can't understand is why the client runs differently or takes longer to crash with the same input.

[frmky]

How confident are you that the memory is stable? msieve really exercises the memory.

The same set of relations:

Msieve v. 1.43
Thu Sep 24 19:51:59 2009
random seeds: f75efbbb 5239874a
factoring 22241756626112025487387232215892970846333943306250217948039483251032302170567514844145619183936201185474527456283676773258556165137875115220508559250721768494720923182129313750686328471794210234430499865105061336147 (215 digits)
no P-1/P+1/ECM available, skipping
commencing number field sieve (215-digit input)
R0: -1224809639974238708818962962512535510581248
R1:  1
A0: -12
A1:  0
A2:  0
A3:  0
A4:  0
A5:  0
A6:  1
skew 1.00, size 2.479471e-12, alpha 1.334283, combined = 2.209296e-13

commencing relation filtering
estimated available RAM is 64454.2 MB
commencing duplicate removal, pass 1

...

found 3509462 hash collisions in 89175031 relations
added 1219066 free relations
commencing duplicate removal, pass 2
found 531 duplicates and 90393566 unique relations

...

commencing 2-way merge
reduce to 23431124 relation sets and 22856526 unique ideals
commencing full merge
memory use: 2889.5 MB
found 12691434 cycles, need 12612726
weight of 12612726 cycles is about 882967842 (70.01/cycle)
distribution of cycle lengths:
1 relations: 1555777
2 relations: 1697997
3 relations: 1670597
4 relations: 1490575
5 relations: 1306310
6 relations: 1105501
7 relations: 926773
8 relations: 753267
9 relations: 595331
10+ relations: 1510598
heaviest cycle: 20 relations
commencing cycle optimization
start with 64865983 relations
pruned 1747798 relations
memory use: 2087.2 MB
distribution of cycle lengths:
1 relations: 1555777
2 relations: 1734853
3 relations: 1735456
4 relations: 1528178
5 relations: 1338266
6 relations: 1121051
7 relations: 932746
8 relations: 746753
9 relations: 581351
10+ relations: 1338295
heaviest cycle: 20 relations
RelProcTime: 33330

commencing linear algebra
read 12612726 cycles
cycles contain 33892207 unique relations
read 33892207 relations
using 20 quadratic characters above 1073741784
building initial matrix
memory use: 4463.3 MB
read 12612726 cycles
matrix is 12612548 x 12612726 (3808.0 MB) with weight 1111885117 (88.16/col)
sparse part has weight 859498235 (68.15/col)
filtering completed in 2 passes
matrix is 12610115 x 12610293 (3807.8 MB) with weight 1111820628 (88.17/col)
sparse part has weight 859481733 (68.16/col)
read 12610293 cycles
matrix is 12610115 x 12610293 (3807.8 MB) with weight 1111820628 (88.17/col)
sparse part has weight 859481733 (68.16/col)
saving the first 48 matrix rows for later
matrix is 12610067 x 12610293 (3630.6 MB) with weight 887328246 (70.37/col)
sparse part has weight 825641182 (65.47/col)
matrix includes 64 packed rows
using block size 65536 for processor cache size 2048 kB
commencing Lanczos iteration (4 threads)
memory use: 3973.0 MB
linear algebra completed 25282 of 12610293 dimensions (0.2%, ETA 1177h37m)

Hmmm, this is significantly less than 12p232 for the same range of q. Perhaps a bit more sieving is in order anyway.

[em99010pepe]

Originally Posted by frmky

How confident are you that the memory is stable? msieve really exercises the memory.

100 % stable. I had some gproms simulations that used 5 GB of memory, ecm too.

[henryzz]

Originally Posted by Andi47

Strange. If I read correctly, msieve started with 89M relations for the first run and 90M relations for the second one. Are you sure you didn't add a forgotten relations file to the msieve.dat file?

free relations made that difference you saw