mersenneforum.org  

Go Back   mersenneforum.org > New To GIMPS? Start Here! > Information & Answers

Reply
 
Thread Tools
Old 2010-10-30, 23:02   #1
pjaj
 
pjaj's Avatar
 
Oct 2010

3·5 Posts
Default Memory allocation

I'm new to GIMPS, so I apologise if this question has already been answered.
I'm running a quad core i7 CPU with Windows 7 Pro 64 bit so, with hyper-threading, there are 8 processors, and hence 8 worker threads.
Is the memory I allocate shared between all 8 workers, or do they each take up to the maximum I've allowed? From the look of it it's the former, so I could give them, say, 6Gb out of my 12Gb to play with, on the other hand I'd not want them to all try to grab their own 6Gb!
I assume that, if I decided to edit an hour's worth of HD video, they would relinquish enough of the memory.
pjaj is offline   Reply With Quote
Old 2010-10-31, 13:02   #2
Mini-Geek
Account Deleted
 
Mini-Geek's Avatar
 
"Tim Sorbera"
Aug 2006
San Antonio, TX USA

17×251 Posts
Default

Quote:
Originally Posted by pjaj View Post
I'm new to GIMPS, so I apologise if this question has already been answered.
I'm running a quad core i7 CPU with Windows 7 Pro 64 bit so, with hyper-threading, there are 8 processors, and hence 8 worker threads.
Is the memory I allocate shared between all 8 workers, or do they each take up to the maximum I've allowed? From the look of it it's the former, so I could give them, say, 6Gb out of my 12Gb to play with, on the other hand I'd not want them to all try to grab their own 6Gb!
I assume that, if I decided to edit an hour's worth of HD video, they would relinquish enough of the memory.
Four things here:
1. Prime95 isn't able to make good use of the virtual cores given by hyper-threading. You should get better overall performance by giving each worker one physical core.
2. As long as you're just using the standard memory options (as opposed to advanced ones by editing config files, which can set each worker's memory allowed), the memory you give it is shared between all workers.
3. The memory settings are only used for P-1/ECM stage 2. When you're doing other parts of a normal first-time LL test (TF and LL), they use relatively little memory, (vague guess: somewhere under 50 MB for each worker, depends on what they're doing) and can't change the amount of memory they need. On a first-time LL assignment, the P-1 stage 2 portion is, roughly, a day or so out of a few weeks.
4. Prime95 does not automatically relinquish memory when other processes need it. Keeping in mind that unless you are requesting P-1 or ECM work for one or more workers, the time spent with high memory usage is quite short, you can manually or automatically limit Prime95's memory usage in certain situations. You can do it manually by stopping high-memory workers when you need more memory available, or setting Prime95's allowed memory lower temporarily. Or you can do it automatically by setting a time period in which the memory available is lower, (you can do this in Options > CPU for simple config, or for more advanced settings, in local.txt) or telling Prime95 to pause or only do low memory work while running a specific program (set in prime.txt). For info on how to use these options, read undoc.txt, specifically the parts on the time syntax, (near the top of the file, can be added to a few other options to specify different values for different times) Memory (in local.txt), PauseWhileRunning (prime.txt), and LowMemWhileRunning (prime.txt). In general, unless the help on the item says to put it in local.txt, put the options in prime.txt.

Last fiddled with by Mini-Geek on 2010-10-31 at 13:04
Mini-Geek is offline   Reply With Quote
Old 2010-10-31, 15:24   #3
pjaj
 
pjaj's Avatar
 
Oct 2010

3×5 Posts
Default

Thanks, that clarifies it.
As a newbie to this I used the defaults and accepted "whatever makes the most sense" and "smart assignment" for the CPU affinity.
It seems that my first set of allocated work is mostly P-1 or ECM as all the workers that can (5), have taken nearly 200Mb each and 3 say that they can't get enough memory and are looking for tasks that need less. Upping the memory allocation still further from 1Gb to 2Gb results in the 5 workers already using about 200Mb upping their allocation to nearer 400Mb, whilst the other 3 carry on with less memory intensive tasks.

However it would seem that is the order of the day, but F1 or Help > help topics both give an error box "Failed to launch help"

Maybe I'm just getting old, but is there any other manual (pdf?) for Prime95 other than undoc.txt and readme.txt?
pjaj is offline   Reply With Quote
Old 2010-10-31, 20:42   #4
Mini-Geek
Account Deleted
 
Mini-Geek's Avatar
 
"Tim Sorbera"
Aug 2006
San Antonio, TX USA

17×251 Posts
Default

Quote:
Originally Posted by pjaj View Post
Thanks, that clarifies it.
As a newbie to this I used the defaults and accepted "whatever makes the most sense" and "smart assignment" for the CPU affinity.
It seems that my first set of allocated work is mostly P-1 or ECM as all the workers that can (5), have taken nearly 200Mb each and 3 say that they can't get enough memory and are looking for tasks that need less. Upping the memory allocation still further from 1Gb to 2Gb results in the 5 workers already using about 200Mb upping their allocation to nearer 400Mb, whilst the other 3 carry on with less memory intensive tasks.
On many first-time assignments, P-1 is one of the first tasks to run. Given the current settings, the described behavior is understandable (I think it's expected; it probably wouldn't be worthwhile for the 3 to replace their less memory intensive tasks with memory intensive ones now, so instead it just gives the extra memory to the 5).
Quote:
Originally Posted by pjaj View Post
Maybe I'm just getting old, but is there any other manual (pdf?) for Prime95 other than undoc.txt and readme.txt?
I don't think there is, at least not a current one.
If you plan to use worker-specific memory settings, you should upgrade to v26 (it's currently like a beta release) instead of v25, because that was fixed recently.

Last fiddled with by Mini-Geek on 2010-10-31 at 20:51 Reason: added reply to earlier part of post
Mini-Geek is offline   Reply With Quote
Old 2011-02-11, 15:30   #5
pjaj
 
pjaj's Avatar
 
Oct 2010

3·5 Posts
Default

OK, I've dropped the number of cores to 4 and it does seem to run quicker, but every time prime95 starts up the Main Thread reports "Too many sections in worktodo.txt". I was hoping that the 4 threads running would pick the suspended work once they finished their current jobs, but no, they go and get new assignments.
Is there any way of forcing the suspended assignments to be picked up, or pausing a running thread once it completes so that I can edit worktodo? Or must I abandon these 4 tasks, which is a pity since they are all about 60% complete. If I must abandon the assignments, how do I do it, I thought I saw the way on one of the web pages, but I can't find it now.
pjaj is offline   Reply With Quote
Old 2011-02-11, 16:20   #6
chalsall
If I May
 
chalsall's Avatar
 
"Chris Halsall"
Sep 2002
Barbados

9,533 Posts
Default

Quote:
Originally Posted by pjaj View Post
Is there any way of forcing the suspended assignments to be picked up, or pausing a running thread once it completes so that I can edit worktodo? Or must I abandon these 4 tasks, which is a pity since they are all about 60% complete. If I must abandon the assignments, how do I do it, I thought I saw the way on one of the web pages, but I can't find it now.
Stop Prime95. Edit your worktodo.txt file to move the assignments for Workers 5 through 8 to Workers 1 through 4. Restart Prime95.

The work will be completed without loss of current effort.
chalsall is offline   Reply With Quote
Old 2011-02-12, 01:59   #7
pjaj
 
pjaj's Avatar
 
Oct 2010

3×5 Posts
Default

Quote:
Originally Posted by chalsall View Post
Stop Prime95. Edit your worktodo.txt file to move the assignments for Workers 5 through 8 to Workers 1 through 4. Restart Prime95.

The work will be completed without loss of current effort.
Yes, but then, when they finish, they will immediately get some more work from the server and my current tasks will languish instead. What I need is a way of running a task until it has completed and then calling a halt. I can then edit the worktodo file to gradually reduce the number of entries till it matches the number of cores.

Last fiddled with by pjaj on 2011-02-12 at 02:00
pjaj is offline   Reply With Quote
Old 2011-02-12, 02:57   #8
sdbardwick
 
sdbardwick's Avatar
 
Aug 2002
North San Diego County

2·11·31 Posts
Default

Quote:
Originally Posted by pjaj View Post
Yes, but then, when they finish, they will immediately get some more work from the server and my current tasks will languish instead. What I need is a way of running
No, then the already requested work will continue. Place the relocated assignments after the first listed assignments in each worker section and the work order will be:
a) current assignment
b) relocated assignments
c) continuation of already requested assignments
d) new assignments

You can also adjust the assignment order in worktodo without upsetting Prime95 or the server.

Edit: Also delete the [worker #5] to [worker #8] entries to stop the complaints about too many sections.

Last fiddled with by sdbardwick on 2011-02-12 at 03:03
sdbardwick is offline   Reply With Quote
Reply

Thread Tools


Similar Threads
Thread Thread Starter Forum Replies Last Post
2 Memory Allocation Errors petrw1 PrimeNet 8 2018-01-13 09:00
"Memory allocation error" JuanTutors Software 25 2009-02-01 21:07
Memory Allocation Error joel893 Software 0 2008-06-03 14:54
Memory Allocation. A_Pryme Software 7 2005-02-12 02:28
Ram allocation JuanTutors Software 15 2004-08-30 01:15

All times are UTC. The time now is 10:30.

Wed Apr 21 10:30:40 UTC 2021 up 13 days, 5:11, 0 users, load averages: 2.82, 2.38, 2.22

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.