mersenneforum.org Prime95 version 29.1 benchmarking -- request for comments
 User Name Remember Me? Password
 Register FAQ Search Today's Posts Mark Forums Read

 2017-03-03, 04:05 #1 Prime95 P90 years forever!     Aug 2002 Yeehaw, FL 7,351 Posts Prime95 version 29.1 benchmarking -- request for comments I'm not sure the Primenet / CPU benchmarks web page is displaying the most helpful information for our users. Prime95 right now sends single-core benchmark timings to the server. Since, prime95 is often memory bandwidth limited, these single-core timings are of little value in comparing CPUs. Should we switch to sending all-core benchmark timings to the server? or perhaps throughput benchmarks? or maybe send all three? OTOH, if we make such a change, the server is not sent the memory configuration (mem. speed, # channels, single-vs-dual rank) to let viewers know why one system is faster than another. One more change I'm considering is only sending data to the server for registered users. Anonymous users are often users making many runs as they tweak their overclock. Does anyone find that web page of use? Perhaps we should delete it? Comments? Other suggestions?
2017-03-03, 08:15   #2
Lorenzo

Aug 2010
Republic of Belarus

2·89 Posts

Quote:
 Originally Posted by Prime95 I'm not sure the Primenet / CPU benchmarks web page is displaying the most helpful information for our users. Prime95 right now sends single-core benchmark timings to the server. Since, prime95 is often memory bandwidth limited, these single-core timings are of little value in comparing CPUs. Should we switch to sending all-core benchmark timings to the server? or perhaps throughput benchmarks? or maybe send all three? OTOH, if we make such a change, the server is not sent the memory configuration (mem. speed, # channels, single-vs-dual rank) to let viewers know why one system is faster than another. One more change I'm considering is only sending data to the server for registered users. Anonymous users are often users making many runs as they tweak their overclock. Does anyone find that web page of use? Perhaps we should delete it? Comments? Other suggestions?
As for me for now benchmark totally not useful. I think it's a good idea to see multi-core benchmark on the page. Also it would be nice to expand FFT diapason up to 32M length. Now it's showing up to 4096K and it's not actual diapason that Prime work on!!! Especial current version of this page not clear for people who working on 332M+.

2017-03-03, 17:14   #3
Mark Rose

"/X\(‘-‘)/X\"
Jan 2013

29·101 Posts

Quote:
 Originally Posted by Prime95 I'm not sure the Primenet / CPU benchmarks web page is displaying the most helpful information for our users.
Agreed.

Quote:
 Prime95 right now sends single-core benchmark timings to the server. Since, prime95 is often memory bandwidth limited, these single-core timings are of little value in comparing CPUs. Should we switch to sending all-core benchmark timings to the server? or perhaps throughput benchmarks? or maybe send all three?
That would be a major improvement.

Quote:
 OTOH, if we make such a change, the server is not sent the memory configuration (mem. speed, # channels, single-vs-dual rank) to let viewers know why one system is faster than another.
This is probably the biggest improvement to be made. A 4770k with DDR4-2400 is vastly faster than a 4770 with DDR3-1600, despite the 100 MHz CPU clock difference. We've also shown dual-rank memory offers an over 10% improvement in throughput.

Quote:
 One more change I'm considering is only sending data to the server for registered users. Anonymous users are often users making many runs as they tweak their overclock.
I'd rather have the anonymous data. We'll collect data from more varied hardware that way, which helps with comparing clock speeds versus architecture. Instead, I have an idea below about verifying data.

Quote:
 Does anyone find that web page of use? Perhaps we should delete it? Comments? Other suggestions?
I think it's a useful page. We should keep it.

One problem is how it handles what is an overclock, which doesn't accommodate turbo speeds. For instance, my i5-6600's have an all core turbo of +300 MHz. So what then is the proper clock of the CPU? 3.3 GHz or 3.6 GHz? I propose we get rid of that.

Instead of the above, a filter based on the machine having done a successful double check would be far more useful. This will encourage those testing/ranking their overclocks to complete a DC as well as proving system stability at whatever clocks they're running at. It wouldn't matter if they're anonymous or not.

 2017-03-05, 04:25 #4 Jayder     Dec 2012 2×139 Posts I would agree that the benchmarks section needs to be overhauled. There are more variables than with GPUs, but I do like the benchmarks on mersenne.ca and I frequently look at them just for fun. However, I do have the complaint that I cannot see the data behind mersenne.ca's benchmarks. For Prime95 benchmarks, I would like an initial page very similar to mersenne.ca's page for LL, with the ability to click on a certain CPU and view the details of that CPU and of the benchmarks for it. That specific processor page is where you would be able to see the impact that different RAM speeds or overclocks make. The value that matters most to me and that I would include on the initial page/overall list of CPUs is maximum throughput for a particular exponent or range of exponents. For my two CPU build, this commonly appears to be with 4 workers each with 7 threads. In other words, don't only measure the throughput where each worker has its own thread, but measure all reasonable possibilities. I would even appreciate it if the benchmarks page had the value comparisons that the mersenne.ca page has. Last fiddled with by Jayder on 2017-03-05 at 04:28
2017-03-05, 23:38   #5
Serpentine Vermin Jar

Jul 2014

2·5·7·47 Posts

Quote:
 Originally Posted by Prime95 I'm not sure the Primenet / CPU benchmarks web page is displaying the most helpful information for our users. Prime95 right now sends single-core benchmark timings to the server. Since, prime95 is often memory bandwidth limited, these single-core timings are of little value in comparing CPUs. Should we switch to sending all-core benchmark timings to the server? or perhaps throughput benchmarks? or maybe send all three? OTOH, if we make such a change, the server is not sent the memory configuration (mem. speed, # channels, single-vs-dual rank) to let viewers know why one system is faster than another. One more change I'm considering is only sending data to the server for registered users. Anonymous users are often users making many runs as they tweak their overclock. Does anyone find that web page of use? Perhaps we should delete it? Comments? Other suggestions?
Whatever the change(s), my vote would be to keep the old data and page for historical purposes and then put all the new benchmarks into a new table, with a new page on the site to show it off.

I just had a feeling that whatever more useful data is collected would be essentially incompatible with what we have now and trying to get them to play nice with each other could be difficult and not really a good comparison.

Fortunately I think once a new benchmark format is worked out, we have a lot of folks with a lot of hardware that could help populate some basic stats on different CPU/memory combos so new users have a decent comparison.

 2017-03-06, 01:47 #6 Mark Rose     "/X\(‘-‘)/X\" Jan 2013 29·101 Posts One additional feature I'd like to see is a way to do a benchmarking run completely with command line switches. The new mprime interface is great, but I can't script it. For instance, I'd like to be able to run something like this: bench.input: Code: 16 1 2048 8192 n y y n y 5 At present that doesn't work: Code: $./mprime -d < bench.input [Main thread Mar 5 20:43] Mersenne number primality test program version 29.1 [Main thread Mar 5 20:43] Optimizing for CPU architecture: Core i3/i5/i7, L2 cache size: 256 KB, L3 cache size: 4 MB Main Menu 1. Test/Primenet 2. Test/Worker threads 3. Test/Status 4. Test/Continue 5. Test/Exit 6. Advanced/Test 7. Advanced/Time 8. Advanced/P-1 9. Advanced/ECM 10. Advanced/Manual Communication 11. Advanced/Unreserve Exponent 12. Advanced/Quit Gimps 13. Options/CPU 14. Options/Preferences 15. Options/Torture Test 16. Options/Benchmark 17. Help/About 18. Help/About PrimeNet Server Your choice: Benchmark type (0 = Throughput, 1 = FFT timings, 2 = Trial factoring) (0): FFT sizes to benchmark Minimum FFT size (in K) (2048): Maximum FFT size (in K) (8192): Benchmark every FFT size in the range (N): CPU cores to benchmark Benchmark using one CPU core (Y): Benchmark using all CPU cores (Y): Benchmark hyperthreading (N): Accept the answers above? (Y): Main Menu 1. Test/Primenet 2. Test/Worker threads 3. Test/Status 4. Test/Stop 5. Test/Exit 6. Advanced/Test 7. Advanced/Time 8. Advanced/P-1 9. Advanced/ECM 10. Advanced/Manual Communication 11. Advanced/Unreserve Exponent 12. Advanced/Quit Gimps 13. Options/CPU 14. Options/Preferences 15. Options/Torture Test 16. Options/Benchmark 17. Help/About 18. Help/About PrimeNet Server Your choice: [Main thread Mar 5 20:43] Starting worker. [Main thread Mar 5 20:43] Stopping all worker threads. Waiting for worker threads to stop. [Work thread Mar 5 20:43] Worker starting [Work thread Mar 5 20:43] Your timings will be written to the results.txt file. [Work thread Mar 5 20:43] Compare your results to other computers at http://www.mersenne.org/report_benchmarks [Work thread Mar 5 20:43] Timing 25 iterations of 2048K FFT length. [Work thread Mar 5 20:43] Execution halted. [Work thread Mar 5 20:43] Worker stopped.$
2017-03-06, 04:35   #7
Prime95
P90 years forever!

Aug 2002
Yeehaw, FL

7,351 Posts

Quote:
 Originally Posted by Madpoo Whatever the change(s), my vote would be to keep the old data and page for historical purposes and then put all the new benchmarks into a new table,.
A new table is not necessary. The table just has two columns: a text field for "what was benchmarked", and a numeric field.

Listening to everyone's comments, I've decided to keep collecting the old data as well as new data: all-cores-FFT-timings and all-cores-throughput. We can figure out what's worth displaying on web pages at a later date.

2017-03-06, 17:10   #8
Serpentine Vermin Jar

Jul 2014

2·5·7·47 Posts

Quote:
 Originally Posted by Prime95 A new table is not necessary. The table just has two columns: a text field for "what was benchmarked", and a numeric field.
I stand corrected (and that proves that I haven't actually looked in the benchmark table, or I would have known that) LOL

2020-01-11, 22:27   #9
Gary

"Gary"
Aug 2015
Texas

2·31 Posts

Quote:
 Originally Posted by Prime95 Listening to everyone's comments, I've decided to keep collecting the old data as well as new data: all-cores-FFT-timings and all-cores-throughput. We can figure out what's worth displaying on web pages at a later date.
Just wondering what the status is on creating a benchmark page for all-cores results. It would be helpful to know what the best reported all-cores performance achieved by others is when setting up a new system.

Suggestion: It would be helpful to add a sentence or two at the top of the benchmark page on how to submit a benchmark result. I eventually discovered that results are automatically submitted to PrimeNet by mprime/prime95 if the benchmark is run a certain way. I think the steps are: Start mprime, make sure it's connected to PrimeNet, select Options/Benchmark, select Benchmark type 1 = FFT timings, accept the default options except select Number of CPU cores = 1. Is this the best / right way?

Also, on the benchmarks page for a certain CPU type, should Stock CPU speed be set to the Base frequency or the Max turbo frequency of the CPU?

Thanks!

 Similar Threads Thread Thread Starter Forum Replies Last Post Prime95 Hardware 32 2016-11-03 03:03 ET_ FermatSearch 5 2016-07-03 10:58 jasong Factoring 6 2006-03-23 05:12 draxen Software 4 2005-04-05 16:41 Cyclamen Persicum Software 2 2004-04-03 14:52

All times are UTC. The time now is 00:52.

Sun Feb 28 00:52:24 UTC 2021 up 86 days, 21:03, 0 users, load averages: 1.95, 1.92, 2.01