mersenneforum.org  

Go Back   mersenneforum.org > Factoring Projects > Cunningham Tables

Reply
 
Thread Tools
Old 2021-11-20, 15:27   #287
EdH
 
EdH's Avatar
 
"Ed Hall"
Dec 2009
Adirondack Mtns

2×2,251 Posts
Default

We're closing in on our primary goal!

Now it's time to show a small portion of my ignorance:

- If A=32 performed so much better returning relations, why was A=30 a better choice? Did the duplicates outweigh the overall return?

- I thought that CPU time was set against an old standard CPU benchmark. If newer CPUs are multiples of that benchmark, why is host #2 (on Seth's page) showing 50% less CPU time over host #3, while showing 50% more productivity? Was that much efficiency really a difference in CPU capabilities?
EdH is offline   Reply With Quote
Old 2021-11-20, 16:18   #288
kruoli
 
kruoli's Avatar
 
"Oliver"
Sep 2017
Porta Westfalica, DE

2×487 Posts
Default

A=32 took much longer to run a single WU (less relations per time) and used much more memory.

Filtering and LA with 1.255B relations gave me (TD 120):
Code:
commencing linear algebra
read 73594760 cycles
cycles contain 241224346 unique relations
read 241224346 relations
using 20 quadratic characters above 4294917295
building initial matrix
memory use: 33714.0 MB
read 73594760 cycles
matrix is 73594580 x 73594760 (27697.0 MB) with weight 8500291408 (115.50/col)
sparse part has weight 6377472282 (86.66/col)
filtering completed in 2 passes
matrix is 73565842 x 73566019 (27695.0 MB) with weight 8499247339 (115.53/col)
sparse part has weight 6377274484 (86.69/col)
matrix starts at (0, 0)
matrix is 73565842 x 73566019 (27695.0 MB) with weight 8499247339 (115.53/col)
sparse part has weight 6377274484 (86.69/col)
saving the first 48 matrix rows for later
matrix includes 64 packed rows
matrix is 73565794 x 73566019 (26838.6 MB) with weight 7031611414 (95.58/col)
sparse part has weight 6299920567 (85.64/col)
using block size 8192 and superblock size 6291456 for processor cache size 65536 kB
commencing Lanczos iteration (16 threads)
memory use: 27168.9 MB
Linear algebra completed 1007 of 73566019 dimensions (0.0%, ETA 3612h 5m)

Last fiddled with by kruoli on 2021-11-20 at 16:20 Reason: Corrected measurement unit.
kruoli is online now   Reply With Quote
Old 2021-11-20, 16:46   #289
charybdis
 
charybdis's Avatar
 
Apr 2020

2B916 Posts
Default

Quote:
Originally Posted by EdH View Post
- If A=32 performed so much better returning relations, why was A=30 a better choice? Did the duplicates outweigh the overall return?
The A/I value determines the size of the sieve region, i.e. how hard we search for relations at each special-Q. Higher A values will always have higher yield but worse rels/sec, but if you use too small a sieve region then the duplication rate will get too high because you'll need to sieve over a very large range of Q.

Quote:
Originally Posted by EdH View Post
- I thought that CPU time was set against an old standard CPU benchmark. If newer CPUs are multiples of that benchmark, why is host #2 (on Seth's page) showing 50% less CPU time over host #3, while showing 50% more productivity? Was that much efficiency really a difference in CPU capabilities?
There's no adjustment for relative CPU speeds, it's literally just plain old CPU-time. This means, for example, that you'll see lower rels/sec if you use hyperthreading. My machines (host #2) do not have HT so this may account for part of the efficiency difference. The rest of the difference is probably a combination of clock speed differences and newer CPUs being more efficient. (If you look at my individual clients you'll see a few of them are substantially faster than the rest. That's not because they've got a higher clock speed - they haven't - it's because they're the newest.)

Quote:
Originally Posted by kruoli View Post
Code:
Linear algebra completed 1007 of 73566019 dimensions (0.0%, ETA 3612h 5m)
Ouch - looks like we might need to go up to 1.4-1.5G relations. Even given the huge matrix, that ETA feels worse than I'd have expected. Were you still sieving on the other 16 threads?
charybdis is offline   Reply With Quote
Old 2021-11-20, 16:53   #290
kruoli
 
kruoli's Avatar
 
"Oliver"
Sep 2017
Porta Westfalica, DE

2×487 Posts
Default

Quote:
Originally Posted by charybdis View Post
Ouch - looks like we might need to go up to 1.4-1.5G relations. Even given the huge matrix, that ETA feels worse than I'd have expected. Were you still sieving on the other 16 threads?
Yes, this was not designed to be a benchmark. I could have used one thread as well, but I was not sure if building the matrix is also parallelised. It looked like it is not.

Our sieving speed (available cores) is still good. We should be able to tackle this quickly.
kruoli is online now   Reply With Quote
Old 2021-11-20, 17:32   #291
EdH
 
EdH's Avatar
 
"Ed Hall"
Dec 2009
Adirondack Mtns

119616 Posts
Default

Thanks for the answers. That does help educate me a bit.

Oliver, are you planning to take the server down when you reach the initial goal? I'm not sure how my scripts will react. If you plan to drop the server during my overnight (which will include your early morning), I will tell all my clients to stop before I head to bed.
EdH is offline   Reply With Quote
Old 2021-11-20, 17:57   #292
kruoli
 
kruoli's Avatar
 
"Oliver"
Sep 2017
Porta Westfalica, DE

97410 Posts
Default

Quote:
Originally Posted by EdH View Post
Oliver, are you planning to take the server down when you reach the initial goal?
No, I am going to wait at least until our experts here agree that we can stop, in that case I would announce my intention and will shut down my server after one hour without no new work sent out or 24 h ignoring existing connections, whichever comes first.
kruoli is online now   Reply With Quote
Old 2021-11-20, 18:05   #293
VBCurtis
 
VBCurtis's Avatar
 
"Curtis"
Feb 2005
Riverside, CA

22×1,319 Posts
Default

CADO should keep running until we get a manageable matrix. 73M at 1.25G relations suggests we have another day or three to go; hopefully filtering can be run once a day until the matrix doesn't shrink much from the day's extra relations.

1.16G -> 87M matrix (default TD)
1.255G -> 73M matrix (TD 120)
So ~100M extra relations dropped 16% from matrix size. Another 50M relations from a day's sieving should reduce another 6% or so; that is, we might lose ~4M dimensions from one more day's sieving.

Projecting from these two data points, 1.4G might yield a matrix around 61-64M. Let's use TD 124 next time, to make filtering work a little harder.
VBCurtis is offline   Reply With Quote
Old 2021-11-20, 18:12   #294
kruoli
 
kruoli's Avatar
 
"Oliver"
Sep 2017
Porta Westfalica, DE

2×487 Posts
Default

This can be done. I will start the next run at around 8 PM UTC. The filtering ran 15 h last time since the machine is busy, so results should be there around 2 PM UTC.
kruoli is online now   Reply With Quote
Old 2021-11-20, 18:31   #295
VBCurtis
 
VBCurtis's Avatar
 
"Curtis"
Feb 2005
Riverside, CA

22×1,319 Posts
Default

Also, you are correct that filtering and matrix-building is single-threaded.
CADO's is multi-threaded, but that doesn't help us here since CADO matrix-solving is so much slower.

If only we knew how to use the results from CADO filtering to run with msieve matrix-solving; on the C207 team sieve I got a 60M matrix from CADO but 72M from msieve.
VBCurtis is offline   Reply With Quote
Old 2021-11-20, 18:33   #296
EdH
 
EdH's Avatar
 
"Ed Hall"
Dec 2009
Adirondack Mtns

2·2,251 Posts
Default

Sounds good. Will you have the server send out a 410 message, or just stop its process? I might pull all my clients on Monday night, if we're still sieving then, due to expecting to be tied up Tuesday.
EdH is offline   Reply With Quote
Old 2021-11-20, 18:36   #297
EdH
 
EdH's Avatar
 
"Ed Hall"
Dec 2009
Adirondack Mtns

2×2,251 Posts
Default

Quote:
Originally Posted by VBCurtis View Post
. . .
If only we knew how to use the results from CADO filtering to run with msieve matrix-solving; on the C207 team sieve I got a 60M matrix from CADO but 72M from msieve.
The CADO-NFS docs explain how, if we'd like to try it:
Quote:
Originally Posted by README.msieve
III) Using msieve linear algebra after CADO-NFS filtering

[should work with CADO-NFS revision aca5658]

up from msieve (svn) revision 891, msieve can read a cycle file produced by
CADO-NFS. To use it, you will have to:

- use CADO-NFS for the filtering. In what follows, let 'prefix' be
the prefix used for all the CADO filenames
- use the CADO 'replay' binary with --for_msieve to produce
a file <prefix>.cyc
- concatenate all the relation files specified by purge.log in
the order specified, and name the file <prefix> in the same
directory as all the other CADO intermediate files. If Msieve was
compiled with zlib support, the files do not have to be uncompressed
- create a <prefix>.fb file with the polynomials in Msieve format
- create worktodo.ini with a single line containing N
- run Msieve LA with

-v -nf <prefix>.fb -s <prefix> -nc2 "cado_filter=1"

The string at the end may get extra options depending on whether
the LA has more tweaking, like using MPI. The .cyc file gets
overwritten during the LA, so re-running the LA does not require
cado_filter=1.
EdH is offline   Reply With Quote
Reply

Thread Tools


Similar Threads
Thread Thread Starter Forum Replies Last Post
Using 16e on smaller numbers fivemack Factoring 3 2017-09-19 08:52
NFS on smaller numbers? skan YAFU 6 2013-02-26 13:57
Bernoulli(200) c204 akruppa Factoring 114 2012-08-20 14:01
checking smaller number fortega Data 2 2005-06-16 22:48
Factoring Smaller Numbers marc Factoring 6 2004-10-09 14:17

All times are UTC. The time now is 15:18.


Mon May 16 15:18:45 UTC 2022 up 32 days, 13:20, 1 user, load averages: 1.07, 1.40, 1.48

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2022, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.

≠ ± ∓ ÷ × · − √ ‰ ⊗ ⊕ ⊖ ⊘ ⊙ ≤ ≥ ≦ ≧ ≨ ≩ ≺ ≻ ≼ ≽ ⊏ ⊐ ⊑ ⊒ ² ³ °
∠ ∟ ° ≅ ~ ‖ ⟂ ⫛
≡ ≜ ≈ ∝ ∞ ≪ ≫ ⌊⌋ ⌈⌉ ∘ ∏ ∐ ∑ ∧ ∨ ∩ ∪ ⨀ ⊕ ⊗ 𝖕 𝖖 𝖗 ⊲ ⊳
∅ ∖ ∁ ↦ ↣ ∩ ∪ ⊆ ⊂ ⊄ ⊊ ⊇ ⊃ ⊅ ⊋ ⊖ ∈ ∉ ∋ ∌ ℕ ℤ ℚ ℝ ℂ ℵ ℶ ℷ ℸ 𝓟
¬ ∨ ∧ ⊕ → ← ⇒ ⇐ ⇔ ∀ ∃ ∄ ∴ ∵ ⊤ ⊥ ⊢ ⊨ ⫤ ⊣ … ⋯ ⋮ ⋰ ⋱
∫ ∬ ∭ ∮ ∯ ∰ ∇ ∆ δ ∂ ℱ ℒ ℓ
𝛢𝛼 𝛣𝛽 𝛤𝛾 𝛥𝛿 𝛦𝜀𝜖 𝛧𝜁 𝛨𝜂 𝛩𝜃𝜗 𝛪𝜄 𝛫𝜅 𝛬𝜆 𝛭𝜇 𝛮𝜈 𝛯𝜉 𝛰𝜊 𝛱𝜋 𝛲𝜌 𝛴𝜎𝜍 𝛵𝜏 𝛶𝜐 𝛷𝜙𝜑 𝛸𝜒 𝛹𝜓 𝛺𝜔