Improved params files for CADO

[VBCurtis]

Hello!
The params for C195 from the post your quoted are good for the first phase of the job, poly select. This phase is about 5% of the total time of the job. You can start that part immediately, if you wish.

I'm in the middle of a midterm week at work, so I might need a day or two to get a complete draft of sieving params for C195 posted here. Before I do:
If you have no experience with factoring software, you really should run some smaller jobs to see how the software runs, how much memory it takes (and how memory scales with job size), etc. A C120 and a C140 are good starter sizes.

Do you understand how long a C195 will take an individual user? Do you have a 64GB-memory machine to run the filtering & matrix steps, or the time to learn how to use CADO for poly select and sieving but msieve for the filtering & matrix steps (msieve will fit on 32GB for this size of job)?

I think the factoring as a service thing was designed for RSA-512 sized keys, which take a modern desktop something like 20-40 core-days to factor. A C193 is a whole lot tougher than that- over 100 times harder.

[charybdis]

Originally Posted by VBCurtis

Do you understand how long a C195 will take an individual user? Do you have a 64GB-memory machine to run the filtering & matrix steps, or the time to learn how to use CADO for poly select and sieving but msieve for the filtering & matrix steps (msieve will fit on 32GB for this size of job)?

I think the factoring as a service thing was designed for RSA-512 sized keys, which take a modern desktop something like 20-40 core-days to factor. A C193 is a whole lot tougher than that- over 100 times harder.

The idea of FAAS is to do the whole thing in the cloud using msieve for the matrix, so access to suitable hardware won't be an issue in itself. The major obstacle is cost. In the original FAAS paper it cost $75 to factor a 512-bit key, so this c193 would set Thornado123 back thousands of dollars. IMO that's a lot to spend for a proof-of-concept that doesn't really advance the state of the art. Note that a 210-digit GNFS was done mainly in the cloud even before the FAAS paper came out.

[thornado123]

Thanks @VBCurtis!
Mhh sounds reasonable, I will start by factoring some smaller numbers to check my setup. If we are talking ~3K$ then it is okay with me.

The big question is what a realistic time estimate would be? (if we use computing power in the a few thousands dollar)

Best regards?

[charybdis]

Originally Posted by thornado123

The big question is what a realistic time estimate would be? (if we use computing power in the a few thousands dollar)

That depends on how many cores you're able to use at a time; I haven't used cloud computing myself so I'm not sure what a realistic number would be. But for a sense of scale, I'm currently running an SNFS job of a similar difficulty (actually probably slightly easier) using ~130 threads. I estimate the step that takes the most CPU-time, the sieving step, will take 20 days in total. Postprocessing adds another ~10% (more if you want to use MPI to do the matrix in a very short wall-clock time), you're doing GNFS which adds another ~5% for polynomial selection, your number is probably a bit harder than mine so let's add ~20% for that. That gets us to ~3600 CPU-days, and that's before considering whether the Amazon EC2 cores are slower than mine.

So you could run ~40 cores for ~100 days, but maybe that's too slow for you. To do 193-digit GNFS in a day or two you'll need thousands of cores for the sieving step, and then for linear algebra you'll need either a similar-sized MPI cluster - not sure if you can get hold of that via the cloud - or more likely a very powerful GPU; see the Msieve GPU linear algebra thread for details on that.

[thornado123]

Originally Posted by charybdis

That depends on how many cores you're able to use at a time; I haven't used cloud computing myself so I'm not sure what a realistic number would be. But for a sense of scale, I'm currently running an SNFS job of a similar difficulty (actually probably slightly easier) using ~130 threads. I estimate the step that takes the most CPU-time, the sieving step, will take 20 days in total. Postprocessing adds another ~10% (more if you want to use MPI to do the matrix in a very short wall-clock time), you're doing GNFS which adds another ~5% for polynomial selection, your number is probably a bit harder than mine so let's add ~20% for that. That gets us to ~3600 CPU-days, and that's before considering whether the Amazon EC2 cores are slower than mine.

So you could run ~40 cores for ~100 days, but maybe that's too slow for you. To do 193-digit GNFS in a day or two you'll need thousands of cores for the sieving step, and then for linear algebra you'll need either a similar-sized MPI cluster - not sure if you can get hold of that via the cloud - or more likely a very powerful GPU; see the Msieve GPU linear algebra thread for details on that.

Mhh exciting! I don't need to do it in days 20-30 days would be fine by me. I will looking into the GPU thread though :)

I will do a test tonight on some smaller numbers and see how my setup preforms. Thanks for the answer!

[orever]

Originally Posted by thornado123

Mhh exciting! I don't need to do it in days 20-30 days would be fine by me. I will looking into the GPU thread though :)

I will do a test tonight on some smaller numbers and see how my setup preforms. Thanks for the answer!

Have you tried to see if http://factordb.com/ has the number you want to factor?

[orever]

Originally Posted by thornado123

Mhh exciting! I don't need to do it in days 20-30 days would be fine by me. I will looking into the GPU thread though :)

I will do a test tonight on some smaller numbers and see how my setup preforms. Thanks for the answer!

Is this another case of a secret number or can you say the number?

Maybe with ECM you can factor faster, or is it an RSA key?

[bur]

Have you ever posted a draft for C155? I thought so, but now I can't find it. Thanks.

[kruoli]

Here is one.

[VBCurtis]

I return home Saturday, I ought to be able to post a faster one Sunday for C150 and C155.

I was nearly done refining C100-C130 params when I left on this trip; C135-C155 I still have more data to take to see if my "improvements" are noise or actually faster. I'm trying harder this round to make parameters change steadily from file to file, which makes it easier to extend to 150-170 digits sizes with reasonable guesses.

[bur]

Thanks, that sounds very promising. I did a C153 with stock CADO parameters so I could compare the times.

Btw, has it ever been considered to share these findings with the CADO team or generally publish them?