ECM Stage 2 RAM

[ATH]

Originally Posted by GP2

If he's using GMP-ECM for stage 2, then I think it responds non-linearly to memory, no? For a given B2 value it wants a certain amount of memory, and if you reduce that memory by a factor of N (by using the -k option to use multiple blocks), it will take considerably longer than N times as much execution time.

So if you're giving 14 GB to a stage 2 that really wants terabytes, you're giving it at least a hundred times less memory than it wants, and it will take maybe many thousands of times longer to complete.

It was "only" ~ 308 GB not terabytes, but still a lot compared to 14 GB:

GMP-ECM 7.0.5-dev [configured with GMP 6.1.2, --enable-asm-redc] [ECM]
Resuming ECM residue saved by ATH@5960X with GMP-ECM 7.0.5-dev on Fri Dec 07 09:21:50 2018
Input number is 2^4007-1 (1207 digits)
Using special division for factor of 2^4007-1
Using B1=160632413-160000000, B2=3200000000000-1200000000000000, polynomial Dickson(30), sigma=0:12427217811893723496
dF=10264320, k=1, d=118107990, d2=17, i0=27077
Can't compute success probabilities for B1 <> B2min
Step 1 took 0ms
Estimated memory usage: 308.19GB

I only took B1 to 1.6e8 but that should not matter since I chose stage2 32e11-12e14 as described in the post earlier.

[storm5510]

Since the discussion has gone to GMP-ECM, I have a question:

GMP-ECM 7.0.5-dev [configured with GMP 6.1.2, --enable-asm-redc] [ECM]

I found a Windows binary of the same version as in the most recent post by ATH, assuming he is referring to Linux. After reading the doc file, and getting it running, I noticed that it does not produce a results file. However, is is extremely fast and can use very large B1 and B2 values.

Question: Is it just something to toy with, or does it serve a purpose more dedicated to this project? Obviously, one must be able to submit their results.

[GP2]

Originally Posted by GP2

As mentioned before, if you want to do stage 1 with mprime/Prime95 and stage 2 wth GMP-ECM, first you need to run mprime/Prime95 with the line GmpEcmHook=1 in your prime.txt file, and you need to set B1 and B2 to the same value (say 3000000) in your ECM2= lines in your worktodo.txt file.

Then mprime/Prime95 will output a bunch of lines starting with N= into your results.txt file, instead of its normal output. If you specified 1000 ECM curves, then there will be 1000 lines with N=.

Note, if B1 and B2 aren't the same value in the worktodo.txt line, then GmpEcmHook=1 will be ignored and mprime/Prime95 will just do both stage 1 and stage 2 all by itself, and will output only the typical single line of the form "Mxxxx completed xxx ECM curves, B1=nnnnnnn, B2=nnnnnnnnn"

I need to clarify the last paragraph.

As I mentioned before, if GmpEcmHook is in effect, then your mprime/Prime95 results.txt file will be full of lines that start with N=

However, the results.txt file will still contain the usual "Mxxxx completed xxx ECM curves, B1=nnnnnnn, B2=nnnnnnnnn" lines, and if a factor is found in stage 1, then it will display the usual:

ECM found a factor in curve #nnn, stage #1
Sigma=xxxxxxxxxxxxxxxx, B1=3000000, B2=3000000.
UID: you/your_machine, 2^nnnn+1 has a factor: nnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnn (ECM curve nnn, B1=3000000, B2=3000000)

The problem is, lines like this will be buried among hundreds or even thousands of lines beginning with N=, depending on how many ECM curves you specified in your worktodo.txt line.

In other words, it's very easy to miss the fact that mprime discovered factors in stage 1!

On Linux you can look for those factors with a command like grep -E -v '^(N=|\[)' results.txt and in Windows PowerShell you can use sls -notmatch '^(N=|\[)' results.txt

If the output is too long and scrolls off the screen, you can append | less to the Linux command, and | Out-Host –Paging to the PowerShell command, and use the space bar to scroll one screenful at a time.

[VBCurtis]

Originally Posted by storm5510

Since the discussion has gone to GMP-ECM, I have a question:

GMP-ECM 7.0.5-dev [configured with GMP 6.1.2, --enable-asm-redc] [ECM]

I found a Windows binary of the same version as in the most recent post by ATH, assuming he is referring to Linux. After reading the doc file, and getting it running, I noticed that it does not produce a results file. However, is is extremely fast and can use very large B1 and B2 values.

Question: Is it just something to toy with, or does it serve a purpose more dedicated to this project? Obviously, one must be able to submit their results.

To make a results file, redirect the screen output to file by ending the command-line with something like ">ecmresults1.txt".

Your commentary that it's extremely fast relies upon choosing a small input. What input number did you test it on? As discussed elsewhere in this thread, GMP-ECM is very fast for small inputs (say, below 2^1500), and rathetr slow for large inputs (over 2^40000).

[storm5510]

Originally Posted by VBCurtis

To make a results file, redirect the screen output to file by ending the command-line with something like ">ecmresults1.txt".

Your commentary that it's extremely fast relies upon choosing a small input. What input number did you test it on? As discussed elsewhere in this thread, GMP-ECM is very fast for small inputs (say, below 2^1500), and rather slow for large inputs (over 2^40000).

I have the below in a batch file so I do not have to type it all every time:

ecm -v %1 %2 < work.txt

What input number? There have been several. I didn't exceed the current PrimeNet ceiling value, (19,999,999).

There are two binaries in the archive I downloaded, ecm.exe and ecmfactor.exe. The latter will not run. A Windows 10 issue, I imagine. I found these at a link ATH had placed in another thread. He has four there, based on different CPU types.

Thank you for replying.

[bur]

I run B1=3M ECM curves on expoenents around 22,000. I noticed all workers use 100MB RAM for stage 2. Is there some setting that limits the usage or is that little memory required?

I have:

Memory=28569 during 7:30-23:30 else 28569
MaxEmergencyMemory=16076
MaxHighMemWorkers=12

in local.txt.


I'm asking because I was surprised by MaxHighMemWorkers before, so maybe there's another hidden option. ;)

[Prime95]

The current ECM stage 2 code cannot put tons of memory to good use.

Strongly suggest using GMP-ECM for stage 2 on exponents below 50,000.

[bur]

Ok, thanks. I like the ease of having just one program work at it, but if it speeds things up considerably. I also noticed, that when yafu/gmp-ecm is running, the CPU gets close to 80 °C and most cores are at 4 GHz, while with mprime the temperature is below 70 °C and most cores are running at 3.2 GHz.

Any idea why that might be?

[ATH]

Originally Posted by bur

Ok, thanks. I like the ease of having just one program work at it, but if it speeds things up considerably. I also noticed, that when yafu/gmp-ecm is running, the CPU gets close to 80 °C and most cores are at 4 GHz, while with mprime the temperature is below 70 °C and most cores are running at 3.2 GHz.

Any idea why that might be?

If it is when mprime is running 22K exponents, then it is because at small exponents (small FFTs) the multithreading is not very efficient, so it is probably not using your entire CPU, unless you run 1 exponent on each core?
Check how much each core is utilized compared to when running yafu/gmp-ecm.

[bur]

It's one core per worker, average utilization when using 11 workers and thus cores is 11.1x on htop, so it seems to use more or less 100% of a core all the time.

With gmp-ecm I have much smaller numbers, around 200 digits. Maybe it's just that gmp-ecm's code is different and causes more heat generation? AVX512 instructions are said to do that, for example (though gmp-ecm doesn't use them, I think),

[VBCurtis]

A guess is that P95 is using AVX-512 so the CPU slows itself, but GMP-ECM isn't waiting on memory so is using the CPU more heavily.
I thought AVX-512 would step down no more than 200Mhz, so my guess isn't a very good one.