mersenneforum.org  

Go Back   mersenneforum.org > Factoring Projects > Cunningham Tables

Reply
 
Thread Tools
Old 2021-11-08, 13:42   #221
EdH
 
EdH's Avatar
 
"Ed Hall"
Dec 2009
Adirondack Mtns

33·167 Posts
Default

Excellent! I was getting ready to post a three column version today, but no extra as you've done. I like the trimming. I've also been playing around with adding a unique clients count per day, but not very aggressively. I'm not really sure it would add anything. I see you changed the 400 to 1000 for comparison. I've been wondering if 400 might be better increased.
EdH is offline   Reply With Quote
Old 2021-11-08, 14:00   #222
kruoli
 
kruoli's Avatar
 
"Oliver"
Sep 2017
Porta Westfalica, DE

17278 Posts
Default

Quote:
Originally Posted by EdH View Post
I've also been playing around with adding a unique clients count per day, but not very aggressively. I'm not really sure it would add anything.
This would likely cause really weird results because I had some clients running with --single in a loop and not having --clientid set. So I have hundreds of "unique" clients.
kruoli is offline   Reply With Quote
Old 2021-11-08, 14:24   #223
EdH
 
EdH's Avatar
 
"Ed Hall"
Dec 2009
Adirondack Mtns

33·167 Posts
Default

Quote:
Originally Posted by kruoli View Post
This would likely cause really weird results because I had some clients running with --single in a loop and not having --clientid set. So I have hundreds of "unique" clients.
I had thought about that as well. My machines are all running --single, but I provide an ID. OTOH, all those unique IDs would show a lot of interest at that time!
EdH is offline   Reply With Quote
Old 2021-11-08, 17:20   #224
VBCurtis
 
VBCurtis's Avatar
 
"Curtis"
Feb 2005
Riverside, CA

25×3×5×11 Posts
Default

Quote:
Originally Posted by EdH View Post
Excellent! I was getting ready to post a three column version today, but no extra as you've done. I like the trimming. I've also been playing around with adding a unique clients count per day, but not very aggressively. I'm not really sure it would add anything. I see you changed the 400 to 1000 for comparison. I've been wondering if 400 might be better increased.
When I requested 100 and 400, I had dropped a zero in my mental conversion from q-range to number of workunits.
Glad you changed 400 to 1000.
Comparing current yield to overall-job-yield tells us most of the relevant info about how yield is falling off, anyway.
VBCurtis is online now   Reply With Quote
Old 2021-11-08, 18:22   #225
EdH
 
EdH's Avatar
 
"Ed Hall"
Dec 2009
Adirondack Mtns

33×167 Posts
Default

Quote:
Originally Posted by VBCurtis View Post
When I requested 100 and 400, I had dropped a zero in my mental conversion from q-range to number of workunits.
Glad you changed 400 to 1000.
Comparing current yield to overall-job-yield tells us most of the relevant info about how yield is falling off, anyway.
Credit Oliver with the change. Should it be even greater?

Curiosity: I have a Colab instance running ATM:
Code:
INFO:root:Running /content/cado-nfs/build/e202773b5d6e/sieve/las
 -poly download/3_748plus.poly -q0 356442000 -A 30 -q1 356444000 -lim0 500000000 -lim1 300000000
 -lpb0 33 -lpb1 34 -mfb0 66 -mfb1 98 -ncurves0 25 -ncurves1 20 -fb1 download/3_748plus.roots1.gz
 -out Colab.eFarm.work/3_748plus.356442000-356444000.gz -t 2 -adjust-strategy 2 -stats-stderr
INFO:root:[Mon Nov  8 16:36:57 2021] Subprocess has PID 12961
The resources show only 2.14GB of 12.69 available, in use. I didn't think RAM use was tied to threads, or at least not too closely. Something odd here?
EdH is offline   Reply With Quote
Old 2021-11-08, 21:08   #226
VBCurtis
 
VBCurtis's Avatar
 
"Curtis"
Feb 2005
Riverside, CA

25×3×5×11 Posts
Default

Quote:
Originally Posted by EdH View Post
Credit Oliver with the change. Should it be even greater?
I didn't think RAM use was tied to threads, or at least not too closely. Something odd here?
I think 1000 and 10000 would be better than 400 and 1000; but it may be that yield just doesn't change much even over 10000 workunits, and just one "most recent yield" calculation has any value.

As for RAM: Seems odd to me. My guess is that CADO doesn't "see" all the available ram due to some colab-specific setting, and is running in minimal-ram configuration. My production runs have used 4,6,8,10 threads per client, and all have similar memory use.
VBCurtis is online now   Reply With Quote
Old 2021-11-08, 21:27   #227
kruoli
 
kruoli's Avatar
 
"Oliver"
Sep 2017
Porta Westfalica, DE

983 Posts
Default

Quote:
Originally Posted by VBCurtis View Post
I think 1000 and 10000 would be better than 400 and 1000 [...].
The values are now at 1000 and 25000. If this is too large I can change it back.
Quote:
Originally Posted by VBCurtis View Post
[...] all have similar memory use.
I saw a small increase of memory footprint when increasing the thread count. I also had a test with 16 threads that was slower than a single worker with 8 cores on a machine with 16 physical cores.
kruoli is offline   Reply With Quote
Old 2021-11-08, 23:50   #228
charybdis
 
charybdis's Avatar
 
Apr 2020

19×37 Posts
Default

Also mystified by the low RAM usage on Colab. It is possible to get CADO to use less memory using -bkthresh1, but this comes at a huge cost in speed, and I don't think it ever gets triggered automatically. Are you sure you didn't catch the RAM usage during initialization when las hadn't started sieving yet?

Also I see the average relations per WU figure has gone up from ~5000 to ~6000. Is somebody running clients with A=31?
charybdis is offline   Reply With Quote
Old 2021-11-09, 00:29   #229
VBCurtis
 
VBCurtis's Avatar
 
"Curtis"
Feb 2005
Riverside, CA

25×3×5×11 Posts
Default

Quote:
Originally Posted by charybdis View Post
Also I see the average relations per WU figure has gone up from ~5000 to ~6000. Is somebody running clients with A=31?
Not guilty on A=31.

Did the yield jump right at Q=300M?
VBCurtis is online now   Reply With Quote
Old 2021-11-09, 00:58   #230
EdH
 
EdH's Avatar
 
"Ed Hall"
Dec 2009
Adirondack Mtns

119D16 Posts
Default

The Colab instances are running rather slow at around 40 minutes per WU, but they are limited to two threads:
Code:
INFO:root:Running /content/cado-nfs/build/e202773b5d6e/sieve/las -poly download/3_748plus.poly -q0 362096000 -A 30 -q1 362098000 -lim0 500000000 -lim1 300000000 -lpb0 33 -lpb1 34 -mfb0 66 -mfb1 98 -ncurves0 25 -ncurves1 20 -fb1 download/3_748plus.roots1.gz -out Colab.eFarm.work/3_748plus.362096000-362098000.gz -t 2 -adjust-strategy 2 -stats-stderr
INFO:root:[Tue Nov  9 00:01:07 2021] Subprocess has PID 18277
. . .
INFO:root:Running /content/cado-nfs/build/e202773b5d6e/sieve/las -poly download/3_748plus.poly -q0 362614000 -A 30 -q1 362616000 -lim0 500000000 -lim1 300000000 -lpb0 33 -lpb1 34 -mfb0 66 -mfb1 98 -ncurves0 25 -ncurves1 20 -fb1 download/3_748plus.roots1.gz -out Colab.eFarm.work/3_748plus.362614000-362616000.gz -t 2 -adjust-strategy 2 -stats-stderr
INFO:root:[Tue Nov  9 00:40:25 2021] Subprocess has PID 18749
As you can see, las was started at 00:40:25 and I took the following check at 00:50:40:
Code:
Waiting to finish the current execution.
Python 3 Google Compute Engine backend

RAM: 3.16 GB/12.69 GB Disk: 46.70 GB/107.72 GB
I just did another check at 00:55:42 and it was identical.

Wouldn't " -bkthresh1" be seen in the invocation for las, if it was used?
EdH is offline   Reply With Quote
Old 2021-11-09, 01:01   #231
charybdis
 
charybdis's Avatar
 
Apr 2020

19·37 Posts
Default

Aha, I think I've got it. I noticed that the averages seemed to be stuck. And indeed:

Quote:
Originally Posted by kruoli View Post
Code:
lastarr=($(cat "$relationCountsLogFile")) <-- should be "tac"?

...

lastarrsize=${#lastarr[*]}
while [ $avcount -lt $lastarrsize ]
  do
    let avsum=${avsum}+${lastarr[$avcount]}
    let avcount=${avcount}+1
    if [ $avcount -eq $smallCount ]
      then
        let avsmall=${avsum}/$smallCount
    fi
    if [ $avcount -eq $bigCount ]
      then
        let avbig=${avsum}/$bigCount
    fi
  done
let avtotal=$avsum/$avcount
Looks an awful lot like it's averaging the first 1000 WUs rather than the last 1000...

Last fiddled with by charybdis on 2021-11-09 at 01:02
charybdis is offline   Reply With Quote
Reply

Thread Tools


Similar Threads
Thread Thread Starter Forum Replies Last Post
Using 16e on smaller numbers fivemack Factoring 3 2017-09-19 08:52
NFS on smaller numbers? skan YAFU 6 2013-02-26 13:57
Bernoulli(200) c204 akruppa Factoring 114 2012-08-20 14:01
checking smaller number fortega Data 2 2005-06-16 22:48
Factoring Smaller Numbers marc Factoring 6 2004-10-09 14:17

All times are UTC. The time now is 18:00.


Tue May 24 18:00:32 UTC 2022 up 40 days, 16:01, 0 users, load averages: 1.71, 1.67, 1.76

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2022, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.

≠ ± ∓ ÷ × · − √ ‰ ⊗ ⊕ ⊖ ⊘ ⊙ ≤ ≥ ≦ ≧ ≨ ≩ ≺ ≻ ≼ ≽ ⊏ ⊐ ⊑ ⊒ ² ³ °
∠ ∟ ° ≅ ~ ‖ ⟂ ⫛
≡ ≜ ≈ ∝ ∞ ≪ ≫ ⌊⌋ ⌈⌉ ∘ ∏ ∐ ∑ ∧ ∨ ∩ ∪ ⨀ ⊕ ⊗ 𝖕 𝖖 𝖗 ⊲ ⊳
∅ ∖ ∁ ↦ ↣ ∩ ∪ ⊆ ⊂ ⊄ ⊊ ⊇ ⊃ ⊅ ⊋ ⊖ ∈ ∉ ∋ ∌ ℕ ℤ ℚ ℝ ℂ ℵ ℶ ℷ ℸ 𝓟
¬ ∨ ∧ ⊕ → ← ⇒ ⇐ ⇔ ∀ ∃ ∄ ∴ ∵ ⊤ ⊥ ⊢ ⊨ ⫤ ⊣ … ⋯ ⋮ ⋰ ⋱
∫ ∬ ∭ ∮ ∯ ∰ ∇ ∆ δ ∂ ℱ ℒ ℓ
𝛢𝛼 𝛣𝛽 𝛤𝛾 𝛥𝛿 𝛦𝜀𝜖 𝛧𝜁 𝛨𝜂 𝛩𝜃𝜗 𝛪𝜄 𝛫𝜅 𝛬𝜆 𝛭𝜇 𝛮𝜈 𝛯𝜉 𝛰𝜊 𝛱𝜋 𝛲𝜌 𝛴𝜎𝜍 𝛵𝜏 𝛶𝜐 𝛷𝜙𝜑 𝛸𝜒 𝛹𝜓 𝛺𝜔