mersenneforum.org

mersenneforum.org (https://www.mersenneforum.org/index.php)
-   EdH (https://www.mersenneforum.org/forumdisplay.php?f=152)
-   -   How I Compile the GPU branch of GMP-ECM in a Colaboratory Session (https://www.mersenneforum.org/showthread.php?t=24887)

EdH 2019-10-26 15:22

How I Compile the GPU branch of GMP-ECM in a Colaboratory Session
 
(Note: I expect to keep the first post of each of these "How I..." threads up-to-date with the latest version. Please read the rest of each thread to see what may have led to the current set of instructions.)

I will take the liberty of expecting readers to already be somewhat familiar with Google's Colaboratory sessions. There are several threads already on Colab and these should be reviewed by interested readers:

[URL="https://mersenneforum.org/showthread.php?t=24646"]Google Colaboratory Notebook?[/URL]
[URL="https://www.mersenneforum.org/showthread.php?t=24818"]GPU72 Notebook Integration...[/URL]
[URL="https://mersenneforum.org/showthread.php?p=527912"]Notebook Instance Reverse SSH and HTTP Tunnels.[/URL]
[URL="https://www.mersenneforum.org/showthread.php?t=24875"]Colab question[/URL]

I do not, as of yet, have a github account, so I have not created an upload of this to github. Others may feel free to do so, if desired. The following is a manner to compile and install GMP and the GPU branch of GMP-ECM into a Colab session, so that ECM can be run making use of a GPU.

For those unfamiliar with the GPU branch of GMP-ECM, only stage 1 is handled by the GPU. All of stage 2 is still only processed by the CPU. Additionally, the stages are not run in parallel. Stage 1 runs on the GPU and then stage 2 runs on the CPU.

More information on the GPU branch of GMP-ECM can be found in the README.gpu document within the ecm main folder.

To use Colab, you need a Gmail account and will be required to log into that account to run a session.

On to the specifics:

Open a [URL="https://colab.research.google.com/notebooks/welcome.ipynb"]Google Colaboratory[/URL] session.
Sign in with your Google/Gmail account info.
Choose New Python3 notebook:
[code]
Menu->File->New Python3 notebook (or within popup)
[/code]Change Runtime to add GPU:[code]
Menu->Runtime->Change runtime type->Hardware accelerator->GPU->SAVE
[/code]Click Connect to start a session.
Edit title from Untitled... to whatever you like.
Paste the following into the Codeblock:
[code]
!chmod 777 /tmp
!apt update
!apt install g++ m4 make subversion libtool autoconf
!mkdir Math
%cd Math
!wget "https://ftp.gnu.org/gnu/gmp/gmp-6.2.0.tar.bz2"
!tar -xf gmp-6.2.0.tar.bz2
!mv gmp-6.2.0 gmp
%cd gmp
!./configure
!make
############################################
# These 4 lines are recommended, but can be commented out or removed! #
############################################
!make check
!echo "Check results, then click in the"
!echo "TextBox below and press Enter to continue. . ."
!read continue in
############################################
!make install
%cd ..
!svn co svn://scm.gforge.inria.fr/svn/ecm/trunk ecm
%cd ecm
!libtoolize
!autoreconf -i
!./configure --enable-gpu --with-cuda-bin=/usr/local/cuda-10.0/bin --with-cuda=/usr/local/cuda-10.0/targets/x86_64-linux --with-cuda-lib=/usr/local/cuda-10.0/targets/x86_64-linux/lib --with-gmp=/usr/local/
!make
############################################
# These 4 lines are recommended, but can be commented out or removed! #
############################################
!make check
!echo "Check results, then click in the"
!echo "TextBox below and press Enter to continue. . ."
!read continue in
############################################
!make install
!echo "############################################"
!echo "############################################"
!echo "(2^835+1)/33" | ecm -gpu 1e4
[/code][B]Note:[/B] There are some checks and pauses written into the code that are recommended, but not strictly necessary. Feel free to comment out or remove those lines in the above code.

Click on the Run cell icon or use CTRL-Enter.
Either watch all the scrolling text or take a break. This will run for a few minutes. If you chose to leave the checks and pauses you can review the results of the GMP and ECM check runs before continuing. To continue at these breaks, you have to click in the text entry box below the message and hit Enter as instructed.

When everything is finished, if all went well, you should have a message similar to the following:
[code]. . .
GMP-ECM 7.0.5-dev [configured with GMP 6.1.2, --enable-asm-redc, --enable-gpu, --enable-assert] [ECM]
Input number is (2^835+1)/33 (250 digits)
Using B1=10000, B2=1873422, sigma=3:1745495596-3:1745496427 (832 curves)
GPU: Block: 32x32x1 Grid: 26x1x1 (832 parallel curves)
Computing 832 Step 1 took 90ms of CPU time / 4044ms of GPU time
Computing 832 Step 2 on CPU took 28406ms
[/code]At this point you can remove the code from the original code block or open a new code block to continue.

Next, in an empty code block, try the following:
[code]
!echo 115367564564210182766242534110944507919869313713243756429 | ecm -gpu 1e4
[/code]You should see a long series of successes:
[code]
GMP-ECM 7.0.5-dev [configured with GMP 6.1.2, --enable-asm-redc, --enable-gpu, --enable-assert] [ECM]
Input number is 115367564564210182766242534110944507919869313713243756429 (57 digits)
Using B1=10000, B2=1873422, sigma=3:2239793235-3:2239794066 (832 curves)
GPU: Block: 32x32x1 Grid: 26x1x1 (832 parallel curves)
GPU: factor 3387679 found in Step 1 with curve 0 (-sigma 3:2239793235)
. . .
GPU: factor 3387679 found in Step 1 with curve 830 (-sigma 3:2239794065)
Computing 832 Step 1 took 62ms of CPU time / 4012ms of GPU time
********** Factor found in step 1: 3387679
Found prime factor of 7 digits: 3387679
Prime cofactor 34055046113935288073705488067477617542827792631251 has 50 digits
[/code]If all ran as expected, you can proceed with more ECM runs using the same format. Again, for more info on running GMP-ECM with GPU support, see the README.gpu document in the main ecm folder.

chalsall 2019-10-26 16:58

[QUOTE=EdH;528999][code]
!apt update
...[/CODE][/QUOTE]

Very nice work!

Dylan figured out that it is advisable to "chmod 777 /tmp" before the "apt update".

This makes your code compatible with Kaggle.

EdH 2019-10-26 17:24

[QUOTE=chalsall;529006]Very nice work!

Dylan figured out that it is advisable to "chmod 777 /tmp" before the "apt update".

This makes your code compatible with Kaggle.[/QUOTE]
Thanks! The step is added. I haven't ventured into Kaggle yet.

chalsall 2019-10-26 19:21

[QUOTE=EdH;529008]Thanks! The step is added. I haven't ventured into Kaggle yet.[/QUOTE]

I hope you give it a go. I don't have the time at the moment to try your code there, but my experience has been that it is relatively trivial to get the Python to run in both environments. Possibly involves some additional "apt install" parms.

And... I just noticed another slight delta to your above code... You're already running as root -- there's no need to "sudo" the "apt-get" command.

EdH 2019-10-26 20:03

[QUOTE=chalsall;529021]I hope you give it a go. I don't have the time at the moment to try your code there, but my experience has been that it is relatively trivial to get the Python to run in both environments. Possibly involves some additional "apt install" parms.

And... I just noticed another slight delta to your above code... You're already running as root -- there's no need to "sudo" the "apt-get" command.[/QUOTE]
Ahh, good catch! Thanks! I probably don't need to use apt-get right after apt either. I must admit to Copy/Paste without a close review. I don't think I need p7zip or zlib1g-dev, either. But, I will have to check now that they are removed.

Dylan14 2019-10-27 00:24

Nice script EdH. But there is one typo: where you have the !make install line for ecm you have an indent, which causes the script to fail. It needs to be inline with the other lines.

EdH 2019-10-27 00:40

[QUOTE=Dylan14;529032]Nice script EdH. But there is one typo: where you have the !make install line for ecm you have an indent, which causes the script to fail. It needs to be inline with the other lines.[/QUOTE]
Thanks! I don't know why those keep showing up. I've already gotten rid of half a dozen.

PhilF 2019-10-28 23:35

This is great! However, I discovered that the number I wanted to work on (M1277) is too large for GPU work.

That sent me on a quest to find suitable numbers that need some ECM thrown at them that were within reach of GPU work, and I found that search difficult.

So where to find some suitable candidates for GPU based ECM curves?

I would like to compare the relative speed of a GPU-enabled GMP-ECM stage 1 with a CPU-based Prime95 stage 1.

VBCurtis 2019-10-29 02:04

[QUOTE=PhilF;529147]

So where to find some suitable candidates for GPU based ECM curves?

I would like to compare the relative speed of a GPU-enabled GMP-ECM stage 1 with a CPU-based Prime95 stage 1.[/QUOTE]

GPU-ECM is limited to 1019 bits. Prime95 is limited to numbers of a certain form. There's not a ton of overlap between them, but I happen to be working on factorizations of k*2^n-1 (Riesel form).
My next two SNFS jobs are:
[code]#expr(13*2^881-1)
466877517878596827431051486024691686855414677675172874855133940880576871475985818271788497478759019442908250077983766404430012687428346626398685854186379459430598338289874292283833496457445239
#expr(13*2^892-1)
18506805396968135611155184891404406020831889021055758328923431924399638720432241766053234118351869077918037177534464815730861966812520562472496745230825092848732924799837750268894252467544668175171[/code]
I believe these can be run by both programs. They've each had about 2t50 done already, and need something close to t55; so curves at 11e7 or bigger are helpful to my efforts.
If you're curious, you can find project status at mklasson.com/factors

PhilF 2019-10-29 02:09

Perfect! That is exactly what I was looking for. I'll do some performance tests and get back with you...

LaurV 2019-10-29 06:19

[QUOTE=EdH;529033]Thanks! I don't know why those keep showing up. I've already gotten rid of half a dozen.[/QUOTE]
That's the forum, gov'nor!

Since last year when Mike changed I don't know what, I have a continuous battle with random spaces appearing in front of code lines in the code tag (therefore mis-aligning tables or space-sensitive code) or empty lines appearing in between paragraphs. Last time I edited somebody's post to align a table header I had to do it 5 or 6 times, because every time some spaces used to appear in front of a different line [U]after I saved[/U] the (correct spaced) post. At the end I was convinced I fixed it, but revisiting the post after few days, some lines inside of the code tag are still mis-aligned. Futile to say, the post still shows I was the last guy to edit it.


All times are UTC. The time now is 08:26.

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.