mersenneforum.org  

Go Back   mersenneforum.org > Great Internet Mersenne Prime Search > PrimeNet

Reply
 
Thread Tools
Old 2017-03-03, 04:05   #1
Prime95
P90 years forever!
 
Prime95's Avatar
 
Aug 2002
Yeehaw, FL

6,833 Posts
Default Prime95 version 29.1 benchmarking -- request for comments

I'm not sure the Primenet / CPU benchmarks web page is displaying the most helpful information for our users.

Prime95 right now sends single-core benchmark timings to the server. Since, prime95 is often memory bandwidth limited, these single-core timings are of little value in comparing CPUs. Should we switch to sending all-core benchmark timings to the server? or perhaps throughput benchmarks? or maybe send all three?

OTOH, if we make such a change, the server is not sent the memory configuration (mem. speed, # channels, single-vs-dual rank) to let viewers know why one system is faster than another.

One more change I'm considering is only sending data to the server for registered users. Anonymous users are often users making many runs as they tweak their overclock.

Does anyone find that web page of use? Perhaps we should delete it?

Comments? Other suggestions?
Prime95 is offline   Reply With Quote
Old 2017-03-03, 08:15   #2
Lorenzo
 
Lorenzo's Avatar
 
Aug 2010
Republic of Belarus

132 Posts
Default

Quote:
Originally Posted by Prime95 View Post
I'm not sure the Primenet / CPU benchmarks web page is displaying the most helpful information for our users.

Prime95 right now sends single-core benchmark timings to the server. Since, prime95 is often memory bandwidth limited, these single-core timings are of little value in comparing CPUs. Should we switch to sending all-core benchmark timings to the server? or perhaps throughput benchmarks? or maybe send all three?

OTOH, if we make such a change, the server is not sent the memory configuration (mem. speed, # channels, single-vs-dual rank) to let viewers know why one system is faster than another.

One more change I'm considering is only sending data to the server for registered users. Anonymous users are often users making many runs as they tweak their overclock.

Does anyone find that web page of use? Perhaps we should delete it?

Comments? Other suggestions?
As for me for now benchmark totally not useful. I think it's a good idea to see multi-core benchmark on the page. Also it would be nice to expand FFT diapason up to 32M length. Now it's showing up to 4096K and it's not actual diapason that Prime work on!!! Especial current version of this page not clear for people who working on 332M+.
Lorenzo is offline   Reply With Quote
Old 2017-03-03, 17:14   #3
Mark Rose
 
Mark Rose's Avatar
 
"/X\(‘-‘)/X\"
Jan 2013
Ͳօɾօղէօ

2·7·199 Posts
Default

Quote:
Originally Posted by Prime95 View Post
I'm not sure the Primenet / CPU benchmarks web page is displaying the most helpful information for our users.
Agreed.

Quote:
Prime95 right now sends single-core benchmark timings to the server. Since, prime95 is often memory bandwidth limited, these single-core timings are of little value in comparing CPUs. Should we switch to sending all-core benchmark timings to the server? or perhaps throughput benchmarks? or maybe send all three?
That would be a major improvement.

Quote:
OTOH, if we make such a change, the server is not sent the memory configuration (mem. speed, # channels, single-vs-dual rank) to let viewers know why one system is faster than another.
This is probably the biggest improvement to be made. A 4770k with DDR4-2400 is vastly faster than a 4770 with DDR3-1600, despite the 100 MHz CPU clock difference. We've also shown dual-rank memory offers an over 10% improvement in throughput.

Quote:
One more change I'm considering is only sending data to the server for registered users. Anonymous users are often users making many runs as they tweak their overclock.
I'd rather have the anonymous data. We'll collect data from more varied hardware that way, which helps with comparing clock speeds versus architecture. Instead, I have an idea below about verifying data.

Quote:
Does anyone find that web page of use? Perhaps we should delete it?

Comments? Other suggestions?
I think it's a useful page. We should keep it.

One problem is how it handles what is an overclock, which doesn't accommodate turbo speeds. For instance, my i5-6600's have an all core turbo of +300 MHz. So what then is the proper clock of the CPU? 3.3 GHz or 3.6 GHz? I propose we get rid of that.

Instead of the above, a filter based on the machine having done a successful double check would be far more useful. This will encourage those testing/ranking their overclocks to complete a DC as well as proving system stability at whatever clocks they're running at. It wouldn't matter if they're anonymous or not.
Mark Rose is offline   Reply With Quote
Old 2017-03-05, 04:25   #4
Jayder
 
Jayder's Avatar
 
"Jared"
Dec 2012

27810 Posts
Default

I would agree that the benchmarks section needs to be overhauled. There are more variables than with GPUs, but I do like the benchmarks on mersenne.ca and I frequently look at them just for fun. However, I do have the complaint that I cannot see the data behind mersenne.ca's benchmarks.

For Prime95 benchmarks, I would like an initial page very similar to mersenne.ca's page for LL, with the ability to click on a certain CPU and view the details of that CPU and of the benchmarks for it. That specific processor page is where you would be able to see the impact that different RAM speeds or overclocks make. The value that matters most to me and that I would include on the initial page/overall list of CPUs is maximum throughput for a particular exponent or range of exponents. For my two CPU build, this commonly appears to be with 4 workers each with 7 threads. In other words, don't only measure the throughput where each worker has its own thread, but measure all reasonable possibilities.

I would even appreciate it if the benchmarks page had the value comparisons that the mersenne.ca page has.

Last fiddled with by Jayder on 2017-03-05 at 04:28
Jayder is offline   Reply With Quote
Old 2017-03-05, 23:38   #5
Madpoo
Serpentine Vermin Jar
 
Madpoo's Avatar
 
Jul 2014

3×1,087 Posts
Default

Quote:
Originally Posted by Prime95 View Post
I'm not sure the Primenet / CPU benchmarks web page is displaying the most helpful information for our users.

Prime95 right now sends single-core benchmark timings to the server. Since, prime95 is often memory bandwidth limited, these single-core timings are of little value in comparing CPUs. Should we switch to sending all-core benchmark timings to the server? or perhaps throughput benchmarks? or maybe send all three?

OTOH, if we make such a change, the server is not sent the memory configuration (mem. speed, # channels, single-vs-dual rank) to let viewers know why one system is faster than another.

One more change I'm considering is only sending data to the server for registered users. Anonymous users are often users making many runs as they tweak their overclock.

Does anyone find that web page of use? Perhaps we should delete it?

Comments? Other suggestions?
Whatever the change(s), my vote would be to keep the old data and page for historical purposes and then put all the new benchmarks into a new table, with a new page on the site to show it off.

I just had a feeling that whatever more useful data is collected would be essentially incompatible with what we have now and trying to get them to play nice with each other could be difficult and not really a good comparison.

Fortunately I think once a new benchmark format is worked out, we have a lot of folks with a lot of hardware that could help populate some basic stats on different CPU/memory combos so new users have a decent comparison.
Madpoo is offline   Reply With Quote
Old 2017-03-06, 01:47   #6
Mark Rose
 
Mark Rose's Avatar
 
"/X\(‘-‘)/X\"
Jan 2013
Ͳօɾօղէօ

53428 Posts
Default

One additional feature I'd like to see is a way to do a benchmarking run completely with command line switches. The new mprime interface is great, but I can't script it. For instance, I'd like to be able to run something like this:

bench.input:
Code:
16
1
2048
8192
n
y
y
n
y
5
At present that doesn't work:

Code:
$ ./mprime -d < bench.input
[Main thread Mar 5 20:43] Mersenne number primality test program version 29.1
[Main thread Mar 5 20:43] Optimizing for CPU architecture: Core i3/i5/i7, L2 cache size: 256 KB, L3 cache size: 4 MB
             Main Menu

         1.  Test/Primenet
         2.  Test/Worker threads
         3.  Test/Status
         4.  Test/Continue
         5.  Test/Exit
         6.  Advanced/Test
         7.  Advanced/Time
         8.  Advanced/P-1
         9.  Advanced/ECM
        10.  Advanced/Manual Communication
        11.  Advanced/Unreserve Exponent
        12.  Advanced/Quit Gimps
        13.  Options/CPU
        14.  Options/Preferences
        15.  Options/Torture Test
        16.  Options/Benchmark
        17.  Help/About
        18.  Help/About PrimeNet Server
Your choice: 
Benchmark type (0 = Throughput, 1 = FFT timings, 2 = Trial factoring) (0): 
FFT sizes to benchmark
Minimum FFT size (in K) (2048): Maximum FFT size (in K) (8192): Benchmark every FFT size in the range (N): 
CPU cores to benchmark
Benchmark using one CPU core (Y): Benchmark using all CPU cores (Y): Benchmark hyperthreading (N): 
Accept the answers above? (Y):       Main Menu

         1.  Test/Primenet
         2.  Test/Worker threads
         3.  Test/Status
         4.  Test/Stop
         5.  Test/Exit
         6.  Advanced/Test
         7.  Advanced/Time
         8.  Advanced/P-1
         9.  Advanced/ECM
        10.  Advanced/Manual Communication
        11.  Advanced/Unreserve Exponent
        12.  Advanced/Quit Gimps
        13.  Options/CPU
        14.  Options/Preferences
        15.  Options/Torture Test
        16.  Options/Benchmark
        17.  Help/About
        18.  Help/About PrimeNet Server
Your choice: [Main thread Mar 5 20:43] 
Starting worker.
[Main thread Mar 5 20:43] Stopping all worker threads.
Waiting for worker threads to stop.
[Work thread Mar 5 20:43] Worker starting
[Work thread Mar 5 20:43] Your timings will be written to the results.txt file.
[Work thread Mar 5 20:43] Compare your results to other computers at http://www.mersenne.org/report_benchmarks
[Work thread Mar 5 20:43] Timing 25 iterations of 2048K FFT length.  
[Work thread Mar 5 20:43] Execution halted.
[Work thread Mar 5 20:43] Worker stopped.
$
Mark Rose is offline   Reply With Quote
Old 2017-03-06, 04:35   #7
Prime95
P90 years forever!
 
Prime95's Avatar
 
Aug 2002
Yeehaw, FL

6,833 Posts
Default

Quote:
Originally Posted by Madpoo View Post
Whatever the change(s), my vote would be to keep the old data and page for historical purposes and then put all the new benchmarks into a new table,.
A new table is not necessary. The table just has two columns: a text field for "what was benchmarked", and a numeric field.

Listening to everyone's comments, I've decided to keep collecting the old data as well as new data: all-cores-FFT-timings and all-cores-throughput. We can figure out what's worth displaying on web pages at a later date.
Prime95 is offline   Reply With Quote
Old 2017-03-06, 17:10   #8
Madpoo
Serpentine Vermin Jar
 
Madpoo's Avatar
 
Jul 2014

3×1,087 Posts
Default

Quote:
Originally Posted by Prime95 View Post
A new table is not necessary. The table just has two columns: a text field for "what was benchmarked", and a numeric field.
I stand corrected (and that proves that I haven't actually looked in the benchmark table, or I would have known that) LOL
Madpoo is offline   Reply With Quote
Old 2020-01-11, 22:27   #9
Gary
 
Gary's Avatar
 
"Gary"
Aug 2015
Texas

3016 Posts
Default

Quote:
Originally Posted by Prime95 View Post
Listening to everyone's comments, I've decided to keep collecting the old data as well as new data: all-cores-FFT-timings and all-cores-throughput. We can figure out what's worth displaying on web pages at a later date.
Just wondering what the status is on creating a benchmark page for all-cores results. It would be helpful to know what the best reported all-cores performance achieved by others is when setting up a new system.

Suggestion: It would be helpful to add a sentence or two at the top of the benchmark page on how to submit a benchmark result. I eventually discovered that results are automatically submitted to PrimeNet by mprime/prime95 if the benchmark is run a certain way. I think the steps are: Start mprime, make sure it's connected to PrimeNet, select Options/Benchmark, select Benchmark type 1 = FFT timings, accept the default options except select Number of CPU cores = 1. Is this the best / right way?

Also, on the benchmarks page for a certain CPU type, should Stock CPU speed be set to the Base frequency or the Max turbo frequency of the CPU?

Thanks!
Gary is offline   Reply With Quote
Reply

Thread Tools


Similar Threads
Thread Thread Starter Forum Replies Last Post
Request for version 28.9 torture testing Prime95 Hardware 32 2016-11-03 03:03
Request for comments ET_ FermatSearch 5 2016-07-03 10:58
Benchmarking problem with Prime95 jasong Factoring 6 2006-03-23 05:12
Request Help - Prime95 Failure draxen Software 4 2005-04-05 16:41
How to use Prime95 For benchmarking and torture testing only Cyclamen Persicum Software 2 2004-04-03 14:52

All times are UTC. The time now is 12:26.

Sat May 30 12:26:00 UTC 2020 up 66 days, 9:59, 3 users, load averages: 2.31, 1.92, 1.80

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.