Non-Prime95 clients talking to PrimeNet... can it be done?

GP2 · 2003-11-29, 19:59

From the Should we buy a backup server? thread and the GIMPS future thread, it seems that non-Prime95 clients such as Mlucas and Glucas can't interface directly with PrimeNet, perhaps because of security and authentication concerns.

In this thread, perhaps we can discuss this issue and come up with solutions.

It would be good to hear whether the proposed PrimeNet v5 will have some way to accomodate non Prime95 clients. If security and authentication are an issue, it would be good to hear from George or Scott what mechanisms could address these concerns. We can also discuss gbvalor's plans for writing a "mini-server" for non-Prime95 clients.

GP2 · 2003-11-29, 20:22

In terms of numbers (installed base), the Apple G4 and G5 are the main computers for which we might want to have a Glucas or Mlucas client that talks directly to PrimeNet.

Can we find a solution similar to what's already done with Prime95? In other words, the number crunching source code is open and public, but the official client that knows the "secret handshake" to talk to PrimeNet is available only in binary form.

The question is, which trusted person compiles the official binary-only client for Glucas or Mlucas. It could be gbvalor, but if security is truly an overriding concern we could go so far as to buy George a used G4 off eBay (similar to the Opteron fundraising) so that he could take gbvalor's source code and link in the PrimeNet interface code.

That doesn't solve the problem for big Unix RISC boxes, but it could be a start.

GP2 · 2003-11-29, 20:29

Although x86 is fairly dominant now, and Unix RISC chips are perhaps fading, the future is still up for grabs. Future clients could run on Apple hardware, on next-generation videogame consoles or graphics cards, on specialized floating-point coprocessor cards (like Clearspeed), or other hosts.

gbvalor has pointed out that non-Prime95 clients need a server to talk to. But ideally there should be only one central server, anything else involves duplicated effort and synchronization issues. So we really should try to find a mechanism to make it all work together with the same server...

Old man PrimeNet · 2003-11-29, 21:51

I replied to this in the other thread about a backup server. The question is already on the v5 team agenda.

GP2 · 2003-11-29, 22:02

Yes, I saw the post
http://www.mersenneforum.org/showthr...5067#post15067
in the other thread.

Given that v5 has been considerably delayed in the past, is there any impediment to making Glucas work with the current v4 server?

If the network protocol stuff works already in the mprime client, a Linux implementation, surely it can't be that hard to port to a BSD-derivative like Apple's OS... or other Unix OSes. In fact, it ought to just compile and run, no?

Is there a reason why we couldn't quickly get a version of Glucas talking to the current PrimeNet? I'm puzzled that Guillermo considers it necessary to go so far as to write a separate "mini-server" for his program to talk to. Is there more here than meets the eye?

As mentioned before, the Glucas client that talks to PrimeNet would be a "trusted binary" (for Apple G4 or other platforms) compiled only by Guillermo (or perhaps George himself), although the general number-crunching source code would still be available... as far as I know, this is exactly what Prime95 and mprime do.

Can we shed some more light on this?

PrimeCruncher · 2003-11-29, 22:28

Quote:

Originally posted by Old man PrimeNet
I replied to this in the other thread about a backup server. The question is already on the v5 team agenda.

Ah, so a list of "to-dos" is being compiled. Well then it sounds to me like the v5 server development project is well underway. As far as making Mlucas and/or Glucas PrimeNet-compatible, probably only communication code needs to be written into the client. I doubt the communication model (client-server) will change with v5 because that would prevent older Prime95 versions from connecting with the server.

gbvalor · 2003-11-29, 22:40

Hello,

I've been out for some hours and there is already a thread about non-prime95 servers. Nice!.

Well, the difference among the x86/prime95/mprime world and the rest is that there are a lot of OS/hardware combinations and then the building of trusted binaries could be non trivial. As example, for Glucas 2.9.0 release at Sourceforge we built more than 30 binaries, and sure we left many architectures out. The places were we built those binaries of course are public, mainly Sourceforge compile Farm and HP Test Drive site. Ernst Mayer also had to build a lot of binaries for their Mlucas releases.

I've been thinking hard about how to join the open source and the trusted computing. And I obviously have not found the solution.

Then, if there is no other solution, we must try the trusted binaries method, although I have to admit I don't like it.

Guillermo.

GP2 · 2003-11-29, 23:12

Quote:

Originally posted by gbvalor
Then, if there is no other solution, we must try the trusted binaries method, although I have to admit I don't like it.

Guillermo.

The important thing is that the actual number crunching and FFT code is open and available for anyone to study and compile, run manual tests, suggest improvements, etc. Making the "true" PrimeNet client itself a trusted binary is a reasonable restriction. Many distributed computing projects like SETI@Home refuse to provide any source code at all.

Also, on many platforms like Apple G4, the average end user will not even know how to use a compiler to compile their own program. And on some Unix RISC platforms the default "cc" compiler might be dumb and produce non-optimal code, while a proprietary highly-optimizing compiler is available from the manufacturer but will not be free and will not be installed on most end-user's machines. So for reasons of end-user idiotproofing and producing the most optimized binaries, it makes plenty of sense to provide binaries even without the security issues.

Surely it would be easier for you to borrow the Linux mprime networking code rather than implement a server from scratch (a huge amount of work, ask Scott). Provided George is willing to share with you the "secret handshake" authentication code for a trusted binary to use in talking to PrimeNet... can you clarify whether this is an issue?

GP2 · 2003-11-29, 23:29

Quote:

Originally posted by gbvalor
Well, the difference among the x86/prime95/mprime world and the rest is that there are a lot of OS/hardware combinations and then the building of trusted binaries could be non trivial. As example, for Glucas 2.9.0 release at Sourceforge we built more than 30 binaries,

But you already decided to make the non-trivial effort to build non-trusted binaries... building trusted binaries would have taken the same amount of compiling effort, no?

In any case, the platform of "main opportunity" for trusted PrimeNet-enabled binaries would be Apple G4/G5... at least we could try the concept there...

Old man PrimeNet · 2003-11-29, 23:41

I tried to persuade the MacLucas & GLucas folks to make trusted binaries, or at least have a few responsible individuals who could make them for various unix flavors, but never got far.

First, the networking support is harder than it looks - George and I spent a year getting it right and even then, looking back, there are some things to do differently for the better. Most of the false starts began with someone saying how they wanted to make the server trivially work with their client (!), at best an incomplete integration; we already had a simple but complete HTTP network API, and had working MPrime network code to use as a first effort-saving step, just as GP2 indicates. Dumbing down the server and introducing weak test state assumptions was a big step in the wrong direction, and broke an adequate uniform API.

The security question often raised the open-source and many-unixes issue Guillermo pointed out. (The many unixes is not so much a compiling effort per se but rather porting issue.) Trusted binaries are not perfect, but they make it harder to hack the interfaces having that 'secret handshake' built in. The manual web forms being open to the public without throttles was the first taste of what to expect with open source codes - and people do 'mine' the server regularly through it. Handing out additional uncontrolled means to do so is just plain asking for trouble. Plus I had an obligation to a commercial entity I had created to keep things tight and cost effective.

The final point to be made is the '90/10' rule - in software we virtually always build what the majority share of a market needs first (in GIMPS case, over 95% for Prime95 as a target for Windows desktops), then consider how much of the remaining fraction of that market (~5% for everything else) is worth chasing immediately considering the cost (effort to implement) is on par with that of the first 95% share. This is an utterly necessary reality check against pouring precious development resources into relatively little if any return on that investment for relatively unpopular clients.

For the v5 system, an open-source Prime95 will have the same principle requirements for transaction throttling as any other client, making trusted binaries less important if not unnecessary. Controlling the assignment/account transaction 'bleeding' can't be much worse that the 'mining' v4 suffers from today. But if we cannot demonstrate good control in prototype v5 testing, we may need to reconsider allowing arbitrary open-sourced untrusted clients.

The '90/10' rule will, however, continue to apply to decisions affecting which target clients are supported first. The upshot again is that from the server's viewpoint (both v4 and v5) a client transacts identically at the protocol level regardless of its platform (client/version-specific assignment decisions & validations, etc., will be implemented below that protocol).

gbvalor · 2003-11-29, 23:48

Quote:

Surely it would be easier for you to borrow the Linux mprime networking code rather than implement a server from scratch (a huge amount of work, ask Scott). Provided George is willing to share with you the "secret handshake" authentication code for a trusted binary to use in talking to PrimeNet... can you clarify whether this is an issue?

When all the excitant news, rings and bells of a new mersenne prime (we hope). I will re-read the visible mprime code to deal with primenet. I remember it was not trivial but I could try.

Quote:

In any case, the platform of "main opportunity" for trusted PrimeNet-enabled binaries would be Apple G4/G5... at least we could try the concept there...

I agree. G4/G5 should be the first test platform.

Now I'm going to bed. It has been a long day.

Guillermo.

2003-11-29, 19:59	#1
GP2 Sep 2003 3²×7×41 Posts	Non-Prime95 clients talking to PrimeNet... can it be done? From the Should we buy a backup server? thread and the GIMPS future thread, it seems that non-Prime95 clients such as Mlucas and Glucas can't interface directly with PrimeNet, perhaps because of security and authentication concerns. In this thread, perhaps we can discuss this issue and come up with solutions. It would be good to hear whether the proposed PrimeNet v5 will have some way to accomodate non Prime95 clients. If security and authentication are an issue, it would be good to hear from George or Scott what mechanisms could address these concerns. We can also discuss gbvalor's plans for writing a "mini-server" for non-Prime95 clients.

2003-11-29, 22:02	#5
GP2 Sep 2003 5027₈ Posts	Yes, I saw the post http://www.mersenneforum.org/showthr...5067#post15067 in the other thread. Given that v5 has been considerably delayed in the past, is there any impediment to making Glucas work with the current v4 server? If the network protocol stuff works already in the mprime client, a Linux implementation, surely it can't be that hard to port to a BSD-derivative like Apple's OS... or other Unix OSes. In fact, it ought to just compile and run, no? Is there a reason why we couldn't quickly get a version of Glucas talking to the current PrimeNet? I'm puzzled that Guillermo considers it necessary to go so far as to write a separate "mini-server" for his program to talk to. Is there more here than meets the eye? As mentioned before, the Glucas client that talks to PrimeNet would be a "trusted binary" (for Apple G4 or other platforms) compiled only by Guillermo (or perhaps George himself), although the general number-crunching source code would still be available... as far as I know, this is exactly what Prime95 and mprime do. Can we shed some more light on this? Last fiddled with by GP2 on 2003-11-29 at 22:05

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
what are we talking about when we talk about Capitalism (not quite R.Carver)	davar55	Soap Box	580	2016-04-25 08:15
How to LMH using Prime95 v25.8 and PrimeNet v5	Uncwilly	Lone Mersenne Hunters	3	2010-03-22 05:08
Thin Clients	moo	Hardware	0	2006-11-14 06:30
MacOS X clients and PrimeNet	schneelocke	Software	9	2005-12-31 02:55
PRP/LLR clients	OmbooHankvald	Prime Sierpinski Project	11	2005-07-13 21:24

2003-11-29, 20:22	#2
GP2 Sep 2003 3²×7×41 Posts	In terms of numbers (installed base), the Apple G4 and G5 are the main computers for which we might want to have a Glucas or Mlucas client that talks directly to PrimeNet. Can we find a solution similar to what's already done with Prime95? In other words, the number crunching source code is open and public, but the official client that knows the "secret handshake" to talk to PrimeNet is available only in binary form. The question is, which trusted person compiles the official binary-only client for Glucas or Mlucas. It could be gbvalor, but if security is truly an overriding concern we could go so far as to buy George a used G4 off eBay (similar to the Opteron fundraising) so that he could take gbvalor's source code and link in the PrimeNet interface code. That doesn't solve the problem for big Unix RISC boxes, but it could be a start.

2003-11-29, 20:29	#3
GP2 Sep 2003 3²·7·41 Posts	Although x86 is fairly dominant now, and Unix RISC chips are perhaps fading, the future is still up for grabs. Future clients could run on Apple hardware, on next-generation videogame consoles or graphics cards, on specialized floating-point coprocessor cards (like Clearspeed), or other hosts. gbvalor has pointed out that non-Prime95 clients need a server to talk to. But ideally there should be only one central server, anything else involves duplicated effort and synchronization issues. So we really should try to find a mechanism to make it all work together with the same server...

2003-11-29, 21:51	#4
Old man PrimeNet Jan 2003 Altitude>12,500 MSL 101 Posts	I replied to this in the other thread about a backup server. The question is already on the v5 team agenda.

2003-11-29, 22:40	#7
gbvalor Aug 2002 3·37 Posts	Hello, I've been out for some hours and there is already a thread about non-prime95 servers. Nice!. Well, the difference among the x86/prime95/mprime world and the rest is that there are a lot of OS/hardware combinations and then the building of trusted binaries could be non trivial. As example, for Glucas 2.9.0 release at Sourceforge we built more than 30 binaries, and sure we left many architectures out. The places were we built those binaries of course are public, mainly Sourceforge compile Farm and HP Test Drive site. Ernst Mayer also had to build a lot of binaries for their Mlucas releases. I've been thinking hard about how to join the open source and the trusted computing. And I obviously have not found the solution. Then, if there is no other solution, we must try the trusted binaries method, although I have to admit I don't like it. Guillermo.

2003-11-29, 23:41	#10
Old man PrimeNet Jan 2003 Altitude>12,500 MSL 101 Posts	I tried to persuade the MacLucas & GLucas folks to make trusted binaries, or at least have a few responsible individuals who could make them for various unix flavors, but never got far. First, the networking support is harder than it looks - George and I spent a year getting it right and even then, looking back, there are some things to do differently for the better. Most of the false starts began with someone saying how they wanted to make the server trivially work with their client (!), at best an incomplete integration; we already had a simple but complete HTTP network API, and had working MPrime network code to use as a first effort-saving step, just as GP2 indicates. Dumbing down the server and introducing weak test state assumptions was a big step in the wrong direction, and broke an adequate uniform API. The security question often raised the open-source and many-unixes issue Guillermo pointed out. (The many unixes is not so much a compiling effort per se but rather porting issue.) Trusted binaries are not perfect, but they make it harder to hack the interfaces having that 'secret handshake' built in. The manual web forms being open to the public without throttles was the first taste of what to expect with open source codes - and people do 'mine' the server regularly through it. Handing out additional uncontrolled means to do so is just plain asking for trouble. Plus I had an obligation to a commercial entity I had created to keep things tight and cost effective. The final point to be made is the '90/10' rule - in software we virtually always build what the majority share of a market needs first (in GIMPS case, over 95% for Prime95 as a target for Windows desktops), then consider how much of the remaining fraction of that market (~5% for everything else) is worth chasing immediately considering the cost (effort to implement) is on par with that of the first 95% share. This is an utterly necessary reality check against pouring precious development resources into relatively little if any return on that investment for relatively unpopular clients. For the v5 system, an open-source Prime95 will have the same principle requirements for transaction throttling as any other client, making trusted binaries less important if not unnecessary. Controlling the assignment/account transaction 'bleeding' can't be much worse that the 'mining' v4 suffers from today. But if we cannot demonstrate good control in prototype v5 testing, we may need to reconsider allowing arbitrary open-sourced untrusted clients. The '90/10' rule will, however, continue to apply to decisions affecting which target clients are supported first. The upshot again is that from the server's viewpoint (both v4 and v5) a client transacts identically at the protocol level regardless of its platform (client/version-specific assignment decisions & validations, etc., will be implemented below that protocol).