BOINC NFS sieving - NFS@Home

[pinhodecarlos]

Hey Jon,

Can you post here your HCN composite list ready to sieve? I think we should feed the grid with these small numbers even if they are 28 or 29 bit. What the rest of you think? I don’t mind post processing them and these can easily be done on 4GB dual core machines. Jarod would be delighted to join us.

Carlos

[jyb]

Originally Posted by pinhodecarlos

Hey Jon,

Can you post here your HCN composite list ready to sieve? I think we should feed the grid with these small numbers even if they are 28 or 29 bit. What the rest of you think? I don’t mind post processing them and these can easily be done on 4GB dual core machines. Jarod would be delighted to join us.

Carlos

At this point the HCNs have all had ECM up to about t47 (with a few having received much more), which means the simple guidelines suggest that GNFS numbers through about 140 digits and SNFS numbers with difficulty through about 211 are ready for sieving.

Strictly speaking, there are about 114 composites that meet those criteria. However, 35 of them would be SNFS numbers with quartic polynomials, and that brings the actual difficulty up to a point where many of them should probably have more ECM before they are sieved.

Some of the SNFS numbers are really easy, like around difficulty 196. IMO that's simply not appropriate for NFS@Home. Anybody with decent hardware can do it themselves. And similarly for GNFS jobs in the mid-130s, of which there are quite a few.

At what difficulty does it become reasonable to use NFS@Home resources? I think it should be at the point where the jobs are getting to be out of reach for typical personal hardware. That's a little bit vague, so I'll be more concrete by saying that I'm comfortable setting a lower limit of 30-bit jobs for the NFS@Home 14e queue. I don't feel the need to be particularly doctrinaire about this, so an occasional 29-bit job might be okay, but I am definitely not in favor of throwing large numbers of small (28- and 29-bit) jobs at the queue. That means that almost all of the HCNs which are ready for NFS are disqualified as being too easy. (I say "almost all" because, as I mentioned above, there are a few numbers of greater difficulty which have gotten sufficient ECM to be ready; I've fed a couple of these to NFS@Home already.)

So this of course gets into questions of utilization. Right now the 14e queue is often starving for work; indeed, as I write this, it has nothing to hand out. So would it be better to keep it fed, even if that means giving it smaller jobs? IMO the answer is no; I would rather the queue run dry for a while than give it lots of small jobs. I have two main reasons for this opinion:

1) Right now, even with semi-frequent empty queues, it looks like those doing post-processing are only just keeping up with the jobs that come through. Unless there's a fresh infusion of volunteers to handle post-processing, then any attempts to keep the queue full may just result in a growing list of jobs awaiting completion.

2) Most people running the grid software now have powerful enough machines to contribute to at least 15e jobs, so there's unlikely to be a large pool of people who are shut out of participation if 14e has no work for them. And right now the 15e queue needs more help. IOW, I would rather see resources get used for chewing through the 15e queue than for doing a whole bunch of jobs on the 14e queue that could just be done by people on their own. From this perspective, the 14e queue being dry on occasion is a feature, not a bug.

I welcome any further discussion of this, especially including contrary opinions. All this is just my personal feeling. Ultimately I'm not in charge of the queues, so I can't stop people from throwing HCNs (or any other numbers) at NFS@Home, if that's what they want to do.

[pinhodecarlos]

Sieve resources are available and if you want the job done than take this window opportunity whilst the 16e queue is not being fed to push forward your project independent of the size of the jobs.

With regards to difficulty of the numbers sieved you can see on the 14e status page where you have a variety of jobs done.

By drying the server people will go away...trust me. At the moment 14e and 16e are at this stage and I’m afraid people will stop running NFS@Home.

Not everybody have fancy computers capable of running the 15e jobs nor the big ones from the 14e in a good time frame so those small jobs support their willing or be part of the NFS@Home post-processing community, which is gratifying.

[VBCurtis]

I agree with jyb for numbers below GNFS-150 in difficulty; I like having jobs available to run personally, and see no need to burn through a ton of really easy work on a grid that could be run entirely at home on a quad. I'll be working this winter on GNFS parameters for 135-155 digits on CADO, and will be happy to run HC numbers of this size (in fact, I'll start in a week or two, as I have more free time soon).

That said, sending a few inputs in GNFS 150-160 range (or corresponding SNFS difficulty) through the 14e queue provides LA work for those with smaller machines, as well as keeping a bit more available grid work for crunchers. It's OK if these are slightly under-ECM'ed; it's not like a t47 when a t49 is called for is some egregious waste of compute time. ECM to 2t45 (~t47) on a GNFS155 is slightly light, but enough for a few numbers to feed 14e if such candidates exist.

[jyb]

Originally Posted by pinhodecarlos

Sieve resources are available and if you want the job done than take this window opportunity whilst the 16e queue is not being fed to push forward your project independent of the size of the jobs.

With regards to difficulty of the numbers sieved you can see on the 14e status page where you have a variety of jobs done.

By drying the server people will go away...trust me. At the moment 14e and 16e are at this stage and I’m afraid people will stop running NFS@Home.

Not everybody have fancy computers capable of running the 15e jobs nor the big ones from the 14e in a good time frame so those small jobs support their willing or be part of the NFS@Home post-processing community, which is gratifying.

The thing is, I don't feel a burning need to "push forward" the HCNs. I find these numbers algebraically interesting, and I enjoy keeping track of their factorizations. I encourage others to work on factoring them because I hope that they too will discover what is interesting about them, and because they form a good pool of numbers with which people can learn to use factoring tools and maybe even push forward the development of algorithms or software (the Cunningham numbers proper are mostly too hard for most people to learn in this way). But in the end it's just another stamp collection. And people who help by running NFS@Home are not learning in the same way as someone who factors the number by him/herself.

Your point about people going away if the queues run dry is a good one. So here's an experiment we should do: how many WU/hr do we see completed in the 15e queue when the 14e queue is full? And how many when the 14e queue is empty? If the 15e queue completes more work when there isn't 14e work to do (as I suspect is the case), then it doesn't seem like people are going away when the 14e queue runs dry. Otherwise we can re-evaluate the question.

[jyb]

Originally Posted by VBCurtis

That said, sending a few inputs in GNFS 150-160 range (or corresponding SNFS difficulty) through the 14e queue provides LA work for those with smaller machines, as well as keeping a bit more available grid work for crunchers. It's OK if these are slightly under-ECM'ed; it's not like a t47 when a t49 is called for is some egregious waste of compute time. ECM to 2t45 (~t47) on a GNFS155 is slightly light, but enough for a few numbers to feed 14e if such candidates exist.

Now that you mention it, there is a GNFS-158 that's already had enough ECM. I'd be happy to send it to the grid, but it needs a polynomial first. I could find one, but it'll be a few days until I can free up some resources for that. Is there a pool of people who are interested in doing polynomial selection on GNFS candidates?

[swellman]

Originally Posted by jyb

Now that you mention it, there is a GNFS-158 that's already had enough ECM. I'd be happy to send it to the grid, but it needs a polynomial first. I could find one, but it'll be a few days until I can free up some resources for that. Is there a pool of people who are interested in doing polynomial selection on GNFS candidates?

You can always ask for help searching for a poly here: https://www.mersenneforum.org/showth...18368&page=143

Folks are willing to poly search, even for some big jobs.

For your reference, the record e-scores for each GNFS size are here: https://www.mersenneforum.org/showpo...5&postcount=86 with updates later in that thread as new records were found.

[jyb]

Originally Posted by swellman

You can always ask for help searching for a poly here: https://www.mersenneforum.org/showth...18368&page=143

Folks are willing to poly search, even for some big jobs.

Looks like the folks in that thread are used to working on much harder numbers. Would they even be interested in something as trifling as a C158?

[pinhodecarlos]

Originally Posted by jyb

Looks like the folks in that thread are used to working on much harder numbers. Would they even be interested in something as trifling as a C158?

Don’t be so negative...you’re worst than myself.

[richs]

Personally, I can only post-process 29 and 30 bit jobs, sometimes 31 bits if I get lucky. I always thought the 14e limit was vaguely GNFS C150, and I have no problem with smaller numbers.

[swellman]

QUEUED AS C252_127_119

C252_127_119 from the XYYXF project is ready for SNFS on 15e.

n: 200393559323943606686635935457538491839164025633034705601625197553945743088916194731736688614521874785295848724216372835216797039938993828927699692906692037017882742587310047413916321981394324138538389530021377015059801031605665547599137121812143479517
# 127^119+119^127, difficulty: 263.59, anorm: 2.46e+038, rnorm: -8.55e+049
# scaled difficulty: 265.52, suggest sieving rational side
# size = 2.480e-013, alpha = 0.000, combined = 4.226e-014, rroots = 0
type: snfs
size: 263
skew: 4.9723
c6: 1
c0: 15113
Y1: -38591013928626616654121403654288432106418519
Y0: 1191446152405248657777607437681912764659201
rlim: 268000000
alim: 268000000
lpbr: 32
lpba: 32
mfbr: 64
mfba: 64
rlambda: 2.8
alambda: 2.8

Test sieving on the -r side with Q in blocks of 2K:

Q=40M   4935
Q=90M   4198
Q=150M  3687
Q=250M  3341
Q=300M  3246

Suggesting a sieving range of 40-290M.