mersenneforum.org

mersenneforum.org (https://www.mersenneforum.org/index.php)
-   No Prime Left Behind (https://www.mersenneforum.org/forumdisplay.php?f=82)
-   -   Opinions wanted on direction of drive 3 (https://www.mersenneforum.org/showthread.php?t=10055)

gd_barnes 2008-03-04 18:55

Opinions wanted on direction of drive 3
 
My original thinking and what I've heard from people is that it would be best to merge drive 3 with drive 1. But I would like to throw out another possibility here:

See how quickly we can get k=300-400 tested to n=600K. I'm currently sieving this k-range for n=600K-1M to P=11T-12T, which will be complete 3rd-4th week in March.

Although we'll have a few k's that we can test for n>600K because others have previously searched that high, if we can get this k-range quickly tested to n=600K, it would open up fully 50 k's for searching n>600K!! :smile: (As far as I know, there are no reservations anywhere for this k-range/n-range.)

Thoughts, opinions, complications, feedback?


Gary

mdettweiler 2008-03-04 19:16

How about we [i]not[/i] merge this with Drive #1, and instead try to pull this drive up to n=600K like you suggested? I'm thinking that way, we can get up to n=600K faster (so we don't have a huge gap in between the leading edge of Drive #3 and whatever drive does the n>600K testing). Not to mention that it will make things a little easier on us LLRnet server folks. :smile:

Thus, long story short, I vote that we keep 300<k<400 and 400<k<1001 separate, but try to generally keep them within n=50K or 100K of each other.

em99010pepe 2008-03-04 19:53

[quote=Anonymous;127784] I vote that we keep 300<k<400 and 400<k<1001 separate..[/quote]

Me too.

henryzz 2008-03-04 19:58

i would say get 300-400 to catch up to 400-1000 and then maybe conbine at n=600k
i dont see any point in keeping them separate longer that we have to

Mini-Geek 2008-03-04 20:02

I think them being two separate drives is okay, but I think that the LLRnet servers shouldn't have a distinction (i.e. servers testing high ranges should reserve from both, as necessary to keep them at about the same n-level, not just from one or the other).

mdettweiler 2008-03-04 20:14

[quote=Mini-Geek;127792]I think them being two separate drives is okay, but I think that the LLRnet servers shouldn't have a distinction (i.e. servers testing high ranges should reserve from both, as necessary to keep them at about the same n-level, not just from one or the other).[/quote]
Unfortunately ,that might be a bit of a hassle for the people involved in administrating the LLRnet servers; thus, I'm thinking that we should still have separate servers for the 300<k<400 and 400<k<1001 range.

em99010pepe 2008-03-04 20:31

[quote=Mini-Geek;127792]I think them being two separate drives is okay, but I think that the LLRnet servers shouldn't have a distinction (i.e. servers testing high ranges should reserve from both, as necessary to keep them at about the same n-level, not just from one or the other).[/quote]

Agree.


[quote=Anonymous;127794]Unfortunately ,that might be a bit of a hassle for the people involved in administrating the LLRnet servers; thus, I'm thinking that we should still have separate servers for the 300<k<400 and 400<k<1001 range.[/quote]

False, it's very easy, you just need to keep adding more work independently of its type but always on the same base (2). One server only, not two but that's my humble and wise opinion. I keep saying that we don't have 100,000 cores but only 100 at our disposal. You people still don't get it?!? Time will give me reason because with the increase of the number of servers the idle ones will also increase, that happened with the CRUS servers, duh!
Sob only has one server, RieselSieve only one server, RS Base 5 two, PSP one...

mdettweiler 2008-03-04 20:46

[quote=em99010pepe;127795]False, it's very easy, you just need to keep adding more work independently of its type but always on the same base (2). One server only, not two but that's my humble and wise opinion. I keep saying that we don't have 100,000 cores but only 100 at our disposal. You people still don't get it?!? Time will give me reason because with the increase of the number of servers the idle ones will also increase, that happened with the CRUS servers, duh!
Sob only has one server, RieselSieve only one server, RS Base 5 two, PSP one...[/quote]
Okay. I wasn't as much thinking of hassles on the part of you or any of the other individual LLRnet server administrators, but instead more along the lines of the work I need to do to turn the LLRnet results files into contiguous lresults-format files. I could do it, but I was thinking it would be a little easier if we kept the two ranges on separate servers.

And yes, I have been keeping in mind our project's current resources--once your servers have been dried and shut down, NPLB shouldn't have any more than 3 servers at any given time. Thus, we shouldn't have the problem of idle servers, as long as we don't go over ~3 servers.

gd_barnes 2008-03-04 23:44

Instead, what makes a project FUN?! :-)
 
We'll get down to fewer servers eventually so let's set aside the server issue for the moment.

Instead, everyone let us know what makes for the most fun project of all! :smile:

The aim of this project intially was to search the ranges as quickly as possible. That quickly changed when I heard several people say that this was one the most fun projects they've been on. I believe that happened largely as a result of the rallies (Thanks Anon/Carlos for bringing those to NPLB!) but I think having sieved files readily available, a good scoring system, and prompt updates of all information on web pages, posts, etc. contributes a large part to that also.

So we've changed...instead of getting the ranges searched quickly, the objective is to keep it as the most fun project of all.

In throwing out the suggestion of not merging drives 1 and 3, keep in mind that if we get a small subset of the big range of k=300-1001 searched to n=600K and begin testing above that, there may be a couple of people here or even people that aren't currently interested that may become very interested if we are searching larger n-ranges. It would also help heavy hitters. With longer testing times, it would be less likely to crash servers.

Yes, many projects have only 1 server but many are BORING projects! I mean no offense to any of them at all because in particular SOB may be one of the best run projects this side of GIMPS. I just can't personally stomach 1000's of secs. per LLR test with virtually only one option of what to search. They just aren't fun to me. I want a project that is fun but is also going places.

If a project is fun for everyone, then people will want to search any and all ranges that we throw out there. And that was the original intent anyway so both objectives are accomplished and everyone has a blast! :smile:


Gary

gd_barnes 2008-03-06 04:35

[quote=Mini-Geek;127792]I think them being two separate drives is okay, but I think that the LLRnet servers shouldn't have a distinction (i.e. servers testing high ranges should reserve from both, as necessary to keep them at about the same n-level, not just from one or the other).[/quote]

[quote=Anonymous;127794]Unfortunately ,that might be a bit of a hassle for the people involved in administrating the LLRnet servers; thus, I'm thinking that we should still have separate servers for the 300<k<400 and 400<k<1001 range.[/quote]


I think what you're both saying here is leading to close to the same thing. Anon, it WOULD be easy to administer if they were at the same n-level. It'd be no more difficult than k=400-1001; only with a few more candidates in each n-range. That was my original thinking but I started this thread to propose a different idea that I thought would be more fun.

I can conclude from this thread 3 things:

1. It's a virtual consensus to keep the drives separate for the time being so that's what we'll do. We can revisit the issue again later.

2. We're almost split on having separate severs for them, with Anon and I in favor of it and Carlos, Mini-geek, and henryzz against with some variations.


Let me throw out what we're looking at regarding servers as it relates to keeping the project fun:

Choice 1; one server:

If we only had one server but the drives were at different n-ranges, it would not be fun for the admins to manage, namely Karsten and Anon, and somewhat me on the back end. In the future, you'd have k=400-1001 candidates at n=400K mixed in with k=300-400 candiates at n=500K or higher. They would have to all be sorted out on the back end and it would be much more error prone. Also, you would not know what n-range your searching at any one time. I think it would be messy.

Choice 2; one server:

If we only had one server but the drives were at the same n-range, this would be very easy for the admins. but would have limited options for searches. We would have:
a. k=300-1001 somewhere around n=425K or so when the drives were able to merge.
b. k=300-400 for n>600K available for ~5-6 k's only that have been previously searched to n=600K. Since we only have the one server, this would have to be a manual-LLR range.


Choice 3; 2 servers:

a. One server for k=300-400 to LLR as quickly as possible to n=600K.
b. One server for k=400-1001 to continue as we are currently.
c. The server from a. changes over to k=300-400 n>600K for a full 50 k's when we reach that limit.

Choice 3 is easy to administer and gives more choices for everyone.

There's one more factor in all of this. The lower k-range of k=300-400 will LLR quite a bit faster at certain n-levels. This is virtually insificant now but will be much more so for n>600K.


My questions to all of you:

Which choice above causes the most people collectively to have fun on the project?

Do you have a better choice than any of the above?


Gary

mdettweiler 2008-03-06 06:11

I'm completely in favor of choice #3. :smile: That way, we avoid all headaches with administration completely, and give users plenty of flexibility for what to crunch (and hence the most fun IMO), yet without spreading our resources too thinly.

Just my $0.02. :smile:

Anon

Mini-Geek 2008-03-06 13:09

I'm okay with Choice 3, but prefer Choice 2.
I guess it really wouldn't matter all that much if people that want to use LLRnet have to choose whether it's k=300-400 or 400-1001. Would it be easier to administrate 3?
I just think that if you don't want to go through the trouble of manual reservations, you wouldn't really care what the k is. It's more complicated to have separate servers, making new people less interested IMHO. Unfortunately, 2 does limit the options for people who run LLRnet [I]and[/I] want to choose.

BTW I think Choice 1 is pretty bad for both the back-end people and clients...why was it even there, other than to make it look like we have more than 2 options? :grin:

mdettweiler 2008-03-06 16:02

[quote=Mini-Geek;127918]I'm okay with Choice 3, but prefer Choice 2.
I guess it really wouldn't matter all that much if people that want to use LLRnet have to choose whether it's k=300-400 or 400-1001. Would it be easier to administrate 3?
I just think that if you don't want to go through the trouble of manual reservations, you wouldn't really care what the k is. It's more complicated to have separate servers, making new people less interested IMHO. Unfortunately, 2 does limit the options for people who run LLRnet [I]and[/I] want to choose.

BTW I think Choice 1 is pretty bad for both the back-end people and clients...why was it even there, other than to make it look like we have more than 2 options? :grin:[/quote]
I see where you're coming from--but I beg to differ as to it being more complicated with 3 servers rather than 2. In fact, we currently have way more than 3 servers (though of course we're trying to clean up some of them), so 3 servers should be a piece of cake for us back-end folks. :wink:

Also, please keep this in mind: since 300<k<400 and 400<k<1001 are going to be separate on the team drive end, it would be a huge mess to try to run them together with LLRnet. It may not look like that at first, but imagine this scenario: Files are merged to make a big 300<k<1001 file for LLRnet. LLRnet crunches it, all fine and dandy. But then when the results files are sent to me, and I have to process them and report them on the respective drive threads...how do I do it? Do I sit at a text editor for 12 hours a day sorting 300<k<400 from 400<k<1001? Conversely, you could keep the results files for both ranges together when dealing with LLRnet, but then we'd have to worry about having the two results files intermingled.

Long story short, I can foresee both choice #1 and #2 becoming hugely messy (though #2 slightly less than #1). To me, the logical way to go seems to be choice #3, since we don't have to worry about any mess (since we won't be putting files from more than one team drive together in the same server at the same time), it would still be quite easy to administrate, and as a side benefit we offer LLRnet users a full gamut of numbers to crunch. :smile:

gd_barnes 2008-03-06 16:14

[quote=Mini-Geek;127792]I think them being two separate drives is okay, but I think that the LLRnet servers shouldn't have a distinction (i.e. servers testing high ranges should reserve from both, as necessary to keep them at about the same n-level, not just from one or the other).[/quote]

[quote=Mini-Geek;127918]
BTW I think Choice 1 is pretty bad for both the back-end people and clients...why was it even there, other than to make it look like we have more than 2 options? :grin:[/quote]

It was there because I inferred that from your original suggestion shown first here. That is, having two separate drives but having server(s) reserve from both. To me, that implies we have 2 drives that are at 2 different testing limits with 1 or 2 (not sure which) LLRnet server(s) not having a distinction and pulling from both drives.

Can you be more specific about what you are impling by your original suggestion? i.e... 2 servers or 1? Drives at the same n-range or not?

If the 2 drives are at the same n-range, we may as well only have one server. If not, I think we need 2 servers for administrative reasons.


Gary

gd_barnes 2008-03-06 16:30

[quote=Anonymous;127928]
Long story short, I can foresee both choice #1 and #2 becoming hugely messy (though #2 slightly less than #1). To me, the logical way to go seems to be choice #3, since we don't have to worry about any mess (since we won't be putting files from more than one team drive together in the same server at the same time), it would still be quite easy to administrate, and as a side benefit we offer LLRnet users a full gamut of numbers to crunch. :smile:[/quote]

No...no...

Choice 2 would be very clean and easy! Actually the easiest of all but IMHO, make for a more BORING project! :smile:

When I said we would merge the DRIVES, I mean that drive 3 would be completely eliminated (actually just completed). There would be no worrying about separating primes/results between the 2 drives. Drive 3 becomes kind of like drive 2 in that it has a specifically defined upper limit (to be determined later).

Upon merging of the drives, there will be a little bit of scoring hassle/setup from Karsten's perspective. We'd probably need to score the 'combined drive' ranges 7/6ths times as much since it would be 7/6ths times as many k's. But we would anticipate that by deciding a specific n-limit that the drives were to merge at so that Karsten would have a very defined rule as to when the scoring changes.

The scoring issue is just a one-time hassle. After that, it would be extremely easy.

Bottom line...

If everyone is looking for easy to administer and understand, choose option 2. Ultimately, we'd need only 1 server total on the project. One to search k=300-1001 up to n=600K. Optionally, we could have a 2nd server to search the few k's for k=300-400 (~5-6 of them) for n>600K that have already been searched that high.

If everyone is looking for more choices and IMHO more fun, choose option 3. Ultimately, we'd need 2 total servers on the project. One for k=300-400 and one for k=400-1001. When k=300-400 completes to n=600K, we'd just load the higher range into the k=300-400 server. No need for a 3rd server except for backup (which is probably a good idea).

I think we're really down to 2 main reasonable choices here. Perhaps I should take a poll on this but I have another question: Should I take a poll to see if I should take a poll? LMAO


Gary

mdettweiler 2008-03-06 17:25

[quote=gd_barnes;127932]No...no...

Choice 2 would be very clean and easy! Actually the easiest of all but IMHO, make for a more BORING project! :smile:

When I said we would merge the DRIVES, I mean that drive 3 would be completely eliminated (actually just completed). There would be no worrying about separating primes/results between the 2 drives. Drive 3 becomes kind of like drive 2 in that it has a specifically defined upper limit (to be determined later).

Upon merging of the drives, there will be a little bit of scoring hassle/setup from Karsten's perspective. We'd probably need to score the 'combined drive' ranges 7/6ths times as much since it would be 7/6ths times as many k's. But we would anticipate that by deciding a specific n-limit that the drives were to merge at so that Karsten would have a very defined rule as to when the scoring changes.

The scoring issue is just a one-time hassle. After that, it would be extremely easy.

Bottom line...

If everyone is looking for easy to administer and understand, choose option 2. Ultimately, we'd need only 1 server total on the project. One to search k=300-1001 up to n=600K. Optionally, we could have a 2nd server to search the few k's for k=300-400 (~5-6 of them) for n>600K that have already been searched that high.

If everyone is looking for more choices and IMHO more fun, choose option 3. Ultimately, we'd need 2 total servers on the project. One for k=300-400 and one for k=400-1001. When k=300-400 completes to n=600K, we'd just load the higher range into the k=300-400 server. No need for a 3rd server except for backup (which is probably a good idea).

I think we're really down to 2 main reasonable choices here. Perhaps I should take a poll on this but I have another question: Should I take a poll to see if I should take a poll? LMAO


Gary[/quote]
Ah, I see what you mean. I thought you were talking about having the drives separate, but with LLRnet reserving ranges from both that would be merged together (which would be quite messy). Sorry, my fault for misunderstanding you. :smile:

With that in mind, as for my opinion on whether we should do choice #2 or #3--I'll still vote for #3. As you said, it makes for a much more fun and interesting project--and hey, isn't that what our goal is now? Thus, #3 is the logical choice. :smile:

As for whether we should take a poll--how about we do a poll if we don't come to a clear consensus here?

henryzz 2008-03-06 17:33

i was looking for an easy solution i think if people are will to spend the extra effort on 3 then do 3 i am fine with that

Flatlander 2008-03-06 18:02

Choice no. 3 is my preference.:smile:

gd_barnes 2008-03-06 19:57

Carlos, Karsten, any others? What do you think of us working towards the following:

1. k=300-400 with one server as we have currently.

2. k=400-1001 with one server after lower ranges are filled in.

3. When drive 2 completes, the servers for it will be eliminated with a possible exception of putting one server temporarily on the higher ranges of the double-check effort.

4. At least one server as backup that does no work except temporarily in 2 situations:
(a) One of our main 2 servers needs an upgrade or goes down for a period of time for whatever reason.
(b) We anticipate an unusually heavy load on one of the 2 servers such as during a rally where Beyond or another person with large resources wants to participate.

The bottom line is that after completing drive 2 and the double-check effort, it should only be very rarely that we're running more than 2 servers.

An option for WHO on the servers: I'm thinking IronBits' for our main 2 servers on both drives with Adam or possibly Carlos or BlisteringSheep 'on call' as backup servers in case we need another temporarily. In other words, they would only run work in 'emergency' situations.


Gary

Beyond 2008-03-06 20:14

2 servers max, and as for the servers the old adage "dance with the one that brung ya" comes to mind.

Mini-Geek 2008-03-06 21:08

[quote=Anonymous;127928]I see where you're coming from--but I beg to differ as to it being more complicated with 3 servers rather than 2. In fact, we currently have way more than 3 servers (though of course we're trying to clean up some of them), so 3 servers should be a piece of cake for us back-end folks. :wink: [/quote]
Actually, I was referring to "Choice 3", not "3" as in "3 servers".
[quote=gd_barnes;127930]It was there because I inferred that from your original suggestion shown first here. That is, having two separate drives but having server(s) reserve from both. To me, that implies we have 2 drives that are at 2 different testing limits with 1 or 2 (not sure which) LLRnet server(s) not having a distinction and pulling from both drives.

Can you be more specific about what you are impling by your original suggestion? i.e... 2 servers or 1? Drives at the same n-range or not?

If the 2 drives are at the same n-range, we may as well only have one server. If not, I think we need 2 servers for administrative reasons.


Gary[/quote]
My suggestion was one "logical" (defined later) server with drives at the same n-range. My confusing wording came from that I was picturing one "logical", if you will, server, which may be made up of multiple physical servers (i.e. multiple servers reserving the same sort of things and effectively acting as one).

I currently think #2 and #3 are good, with a slight preference to #2 (less of a preference to it than before, after reading posts since my last one).

kar_bon 2008-03-06 23:51

i prefer 2 servers too.
one for 300<k<400 and 400<k<1002 each.
so it's the easiest way to switch around and the best way for admins to manage and for Anon/me to evaluate the results.

IronBits 2008-03-07 00:07

Does two different ports == two different servers ?
or you talking two different servers with multiple ports?

kar_bon 2008-03-07 00:14

[QUOTE=IronBits;127988]Does two different ports == two different servers ?
or you talking two different servers with multiple ports?[/QUOTE]

one port is one server because different ports behave like different server (k/n-range)!

tnerual 2008-03-07 00:16

[QUOTE=IronBits;127988]Does two different ports == two different servers ?
or you talking two different servers with multiple ports?[/QUOTE]

one server = one port ...

so the best thing is to put the 2 llrnet server on two physically different internet connection (not even the same ISP)

now, looking at Gary's choice ... i will choose (long term vision)

2 server, one for k= 300-400 (n will increase rapidly), the other for k = 400-1001 (n will increase slowly)

short term vision is 3 server (2 above and 1 for cleaning up Gary's mess)

gd_barnes 2008-03-07 05:33

Excellent everyone! Thanks for expressing your opinions. It will be 2 servers, 1 each on k=300-400 and k=400-1001, after the dust settles in the future.

As Tnerual said, we'll have 3 servers currently after Carlos' current servers run dry. Then when "Gary's mess" and the double-checking are done, we'll have two.

This will be FUN! I'd like to put a 'slight' priority on getting k=300-400 up to n=600K but still keep going on k=400-1001. Perhaps have it searched to n=~450K by the time we're hitting n=600K on k=300-400.

We'll plan to have rallies every 2-3 weeks on whichever range we think needs testing at the time. Perhaps even a 48-hour rally at some point! :smile:

EDIT: THREAD IS BEING UNSTICKIED NOW since we have enough stickies to last us a while.


Gary

mdettweiler 2008-03-07 05:40

Okay, sounds good to me! :smile:

As for a 48-hour rally: yeah, that sounds like it might be a lot of fun. :smile: In fact, one of the original SR5 rallies that ours were inspired by was actually a 48-hour one.

BTW, I fixed a typo in the thread title: 'Opionions' --> 'Opinions'. :smile:


All times are UTC. The time now is 12:13.

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2022, Jelsoft Enterprises Ltd.