Running multiple ecm's

[jwaltos]

I used that B1 to deep dive into certain numbers.
Numerous curves have been run at smaller B1`s without success.

[henryzz]

Originally Posted by wreck

I'm not sure why you use B1=50e9, but , here is some of my experience.
For curiocity / test reason, I ran 1 curve at B1=25e9 (t80) on 10,323- c271,
which takes about one week.
I use -maxmem 12000 option, processor is i7, operating system is
Ubuntu 14.04 LTS 64 bit.
Currently, the biggest number factored by gnfs is RSA768, a c232.
I guess you should finish t75 first, that means using B1=76e8.

The idea of finishing a txx before starting tyy is not borne out by statistics. The probability of finding a factor with a certain B1 drops well before enough curves are done for a 1-e^-1 probability of finding the factor. Enough curves should be run at smaller sizes in order that time isn't wasted using large curves to find small factors but the vast majority of the effort should be at larger curves than are usually recommended. This is borne out by the results of Bayesian analysis of ecm probabilities.

[storm5510]

Originally Posted by johnadam74

Hi,

I'm new at running ecm and I need some advice.

I have a 6 core windows computer and I'm running 10K curves with a B1=110M using a pre-compiled 7.0 version of ecm.

My command is ecm -v -chkpnt C20146980.txt -c 10000 110000000.

Should I be running 6 different instances of that program to reduce the running time?

Any suggestions that you have would be great.

Thanks,


John

This is very interesting. I am running v7.0.5-dev on Windows 10 v1903. Everything goes to one thread only, a physical core. This leaves three cores and 4 threads unused. CPU utilization is 15%. I run the B1's with Prime95 and the switch to save the residues for GMP-ECM. It can run a B2 of 8e12 in 16 minutes using just over 5.6 GB or RAM. This if for very small exponents with no factors found. That's all.

[PhilF]

Originally Posted by storm5510

This is very interesting. I am running v7.0.5-dev on Windows 10 v1903. Everything goes to one thread only, a physical core. This leaves three cores and 4 threads unused. CPU utilization is 15%. I run the B1's with Prime95 and the switch to save the residues for GMP-ECM. It can run a B2 of 8e12 in 16 minutes using just over 5.6 GB or RAM. This if for very small exponents with no factors found. That's all.

If you run multiple instances of Prime95 doing stage 1 curves, be sure to set the affinity to run on different cores in local.txt. For example:

[Worker #1]
Affinity=1

[storm5510]

Originally Posted by PhilF

If you run multiple instances of Prime95 doing stage 1 curves, be sure to set the affinity to run on different cores in local.txt. For example:

[Worker #1]
Affinity=1

Multiple instances. Stage 1 uses minimal RAM. I run a single instance and use all physical cores: Affinity=0,2,4,6. It may be possible to run four workers and and assign the affinity accordingly. Worker 1 would be Affinity=0, worker 2 would be Affinity=2, and so on. This would only serve to slow the process by running them split. I don't understand how this would translate over to GMP-ECM though.

[PhilF]

Originally Posted by storm5510

Multiple instances. Stage 1 uses minimal RAM. I run a single instance and use all physical cores: Affinity=0,2,4,6. It may be possible to run four workers and and assign the affinity accordingly. Worker 1 would be Affinity=0, worker 2 would be Affinity=2, and so on. This would only serve to slow the process by running them split. I don't understand how this would translate over to GMP-ECM though.

ECM testing on Prime95 is single-threaded, and can't take advantage of those extra cores. I think you will find if you assign a single core to Prime95, assuming it is running Stage 1 ECM curves only, it will be nearly as fast as it is now. If you then run multiple instances of Prime 95 and assign each to its own physical core, each instance will have very little effect on speed of the others (as long as all are running Stage 1 ECM curves), and your overall processor utilization will increase proportionately.

[ATH]

Also Prime95 multi threading is very inefficient at very small FFT sizes (very small exponents) .

[storm5510]

Originally Posted by PhilF

ECM testing on Prime95 is single-threaded, and can't take advantage of those extra cores. I think you will find if you assign a single core to Prime95, assuming it is running Stage 1 ECM curves only, it will be nearly as fast as it is now. If you then run multiple instances of Prime 95 and assign each to its own physical core, each instance will have very little effect on speed of the others (as long as all are running Stage 1 ECM curves), and your overall processor utilization will increase proportionately.

What I did was configured Prime95 for two workers. I gave each worker two cores. The throughput is the same.

I did the same with my older HP workstation. It's an i5 which does not hyper-thread. I have a Dell laptop which has a two-core i5. I am trying it now with a single worker.

When I got up early this morning, the two desktops had produced 68 completed stage one tests. It will take about 10 hours for GMP-ECM to run the stage two's. I do not run Prime95 on this i7 when GMP-ECM is running.

I have my GMP-ECM parameters in a small batch file. This way, I do not have to remember them.

[PhilF]

Originally Posted by storm5510

What I did was configured Prime95 for two workers. I gave each worker two cores. The throughput is the same.

I don't think so. If you ran 4 separate Prime95's, each with only one worker, doing Stage 1 ECM curves on only one specifically assigned physical core, you would get twice (or maybe even 4 times) the throughput compared to what you are currently getting.

Of course, this may not do you much good unless you have enough GMP-ECMs running elsewhere to keep up with the output from 4 copies of Prime95 cranking out Stage 1 curve residues.

[storm5510]

Originally Posted by PhilF

I don't think so. If you ran 4 separate Prime95's, each with only one worker, doing Stage 1 ECM curves on only one specifically assigned physical core, you would get twice (or maybe even 4 times) the throughput compared to what you are currently getting.

Of course, this may not do you much good unless you have enough GMP-ECMs running elsewhere to keep up with the output from 4 copies of Prime95 cranking out Stage 1 curve residues.

Perhaps not, but I'm not going to complain. I have nearly 100 B2's to run at the moment. That's plenty for now.

[WraithX]

Originally Posted by schickel

The other choice would be to install ECMNET with a server and 6 client instances running. I prefer doing it this way, since I can load more than one number in just in case a factor pops out before I expect it.

Just as an FYI: You can load multiple numbers into a single text file and feed that into ecm.py. It will work on the numbers one at a time, moving to the next one in the file once a factor is found, or once the requested number of curves has finished with no factor found.

The command looks the same, too: python ecm.py [options] [B1] < numbers.txt