mersenneforum.org  

Go Back   mersenneforum.org > Extra Stuff > Blogorrhea > EdH

Reply
 
Thread Tools
Old 2019-10-26, 15:22   #1
EdH
 
EdH's Avatar
 
"Ed Hall"
Dec 2009
Adirondack Mtns

1101001011002 Posts
Default How I Compile the GPU branch of GMP-ECM in a Colaboratory Session

(Note: I expect to keep the first post of each of these "How I..." threads up-to-date with the latest version. Please read the rest of each thread to see what may have led to the current set of instructions.)

I will take the liberty of expecting readers to already be somewhat familiar with Google's Colaboratory sessions. There are several threads already on Colab and these should be reviewed by interested readers:

Google Colaboratory Notebook?
GPU72 Notebook Integration...
Notebook Instance Reverse SSH and HTTP Tunnels.
Colab question

I do not, as of yet, have a github account, so I have not created an upload of this to github. Others may feel free to do so, if desired. The following is a manner to compile and install GMP and the GPU branch of GMP-ECM into a Colab session, so that ECM can be run making use of a GPU.

For those unfamiliar with the GPU branch of GMP-ECM, only stage 1 is handled by the GPU. All of stage 2 is still only processed by the CPU. Additionally, the stages are not run in parallel. Stage 1 runs on the GPU and then stage 2 runs on the CPU.

More information on the GPU branch of GMP-ECM can be found in the README.gpu document within the ecm main folder.

To use Colab, you need a Gmail account and will be required to log into that account to run a session.

On to the specifics:

Open a Google Colaboratory session.
Sign in with your Google/Gmail account info.
Choose New Python3 notebook:
Code:
Menu->File->New Python3 notebook (or within popup)
Change Runtime to add GPU:
Code:
Menu->Runtime->Change runtime type->Hardware accelerator->GPU->SAVE
Click Connect to start a session.
Edit title from Untitled... to whatever you like.
Paste the following into the Codeblock:
Code:
!chmod 777 /tmp
!apt update
!apt install g++ m4 make subversion libtool autoconf
!mkdir Math
%cd Math
!wget "https://ftp.gnu.org/gnu/gmp/gmp-6.2.0.tar.bz2"
!tar -xf gmp-6.2.0.tar.bz2
!mv gmp-6.2.0 gmp
%cd gmp
!./configure
!make
############################################
# These 4 lines are recommended, but can be commented out or removed! #
############################################
!make check
!echo "Check results, then click in the"
!echo "TextBox below and press Enter to continue. . ."
!read continue in
############################################
!make install
%cd ..
!svn co svn://scm.gforge.inria.fr/svn/ecm/trunk ecm
%cd ecm
!libtoolize
!autoreconf -i
!./configure --enable-gpu --with-cuda-bin=/usr/local/cuda-10.0/bin --with-cuda=/usr/local/cuda-10.0/targets/x86_64-linux --with-cuda-lib=/usr/local/cuda-10.0/targets/x86_64-linux/lib --with-gmp=/usr/local/
!make
############################################
# These 4 lines are recommended, but can be commented out or removed! #
############################################
!make check
!echo "Check results, then click in the"
!echo "TextBox below and press Enter to continue. . ."
!read continue in
############################################
!make install
!echo "############################################"
!echo "############################################"
!echo "(2^835+1)/33" | ecm -gpu 1e4
Note: There are some checks and pauses written into the code that are recommended, but not strictly necessary. Feel free to comment out or remove those lines in the above code.

Click on the Run cell icon or use CTRL-Enter.
Either watch all the scrolling text or take a break. This will run for a few minutes. If you chose to leave the checks and pauses you can review the results of the GMP and ECM check runs before continuing. To continue at these breaks, you have to click in the text entry box below the message and hit Enter as instructed.

When everything is finished, if all went well, you should have a message similar to the following:
Code:
. . .
GMP-ECM 7.0.5-dev [configured with GMP 6.1.2, --enable-asm-redc, --enable-gpu, --enable-assert] [ECM]
 Input number is (2^835+1)/33 (250 digits)
Using B1=10000, B2=1873422, sigma=3:1745495596-3:1745496427 (832 curves)
GPU: Block: 32x32x1 Grid: 26x1x1 (832 parallel curves)
Computing 832 Step 1 took 90ms of CPU time / 4044ms of GPU time
Computing 832 Step 2 on CPU took 28406ms
At this point you can remove the code from the original code block or open a new code block to continue.

Next, in an empty code block, try the following:
Code:
!echo 115367564564210182766242534110944507919869313713243756429 | ecm -gpu 1e4
You should see a long series of successes:
Code:
GMP-ECM 7.0.5-dev [configured with GMP 6.1.2, --enable-asm-redc, --enable-gpu, --enable-assert] [ECM]
Input number is 115367564564210182766242534110944507919869313713243756429 (57 digits)
Using B1=10000, B2=1873422, sigma=3:2239793235-3:2239794066 (832 curves)
GPU: Block: 32x32x1 Grid: 26x1x1 (832 parallel curves)
GPU: factor 3387679 found in Step 1 with curve 0 (-sigma 3:2239793235)
. . .
GPU: factor 3387679 found in Step 1 with curve 830 (-sigma 3:2239794065)
Computing 832 Step 1 took 62ms of CPU time / 4012ms of GPU time
********** Factor found in step 1: 3387679
Found prime factor of 7 digits: 3387679
Prime cofactor 34055046113935288073705488067477617542827792631251 has 50 digits
If all ran as expected, you can proceed with more ECM runs using the same format. Again, for more info on running GMP-ECM with GPU support, see the README.gpu document in the main ecm folder.

Last fiddled with by EdH on 2020-06-02 at 11:02
EdH is offline   Reply With Quote
Old 2019-10-26, 16:58   #2
chalsall
If I May
 
chalsall's Avatar
 
"Chris Halsall"
Sep 2002
Barbados

2×4,643 Posts
Default

Quote:
Originally Posted by EdH View Post
Code:
!apt update
...
Very nice work!

Dylan figured out that it is advisable to "chmod 777 /tmp" before the "apt update".

This makes your code compatible with Kaggle.
chalsall is offline   Reply With Quote
Old 2019-10-26, 17:24   #3
EdH
 
EdH's Avatar
 
"Ed Hall"
Dec 2009
Adirondack Mtns

D2C16 Posts
Default

Quote:
Originally Posted by chalsall View Post
Very nice work!

Dylan figured out that it is advisable to "chmod 777 /tmp" before the "apt update".

This makes your code compatible with Kaggle.
Thanks! The step is added. I haven't ventured into Kaggle yet.
EdH is offline   Reply With Quote
Old 2019-10-26, 19:21   #4
chalsall
If I May
 
chalsall's Avatar
 
"Chris Halsall"
Sep 2002
Barbados

221068 Posts
Default

Quote:
Originally Posted by EdH View Post
Thanks! The step is added. I haven't ventured into Kaggle yet.
I hope you give it a go. I don't have the time at the moment to try your code there, but my experience has been that it is relatively trivial to get the Python to run in both environments. Possibly involves some additional "apt install" parms.

And... I just noticed another slight delta to your above code... You're already running as root -- there's no need to "sudo" the "apt-get" command.
chalsall is offline   Reply With Quote
Old 2019-10-26, 20:03   #5
EdH
 
EdH's Avatar
 
"Ed Hall"
Dec 2009
Adirondack Mtns

22×3×281 Posts
Default

Quote:
Originally Posted by chalsall View Post
I hope you give it a go. I don't have the time at the moment to try your code there, but my experience has been that it is relatively trivial to get the Python to run in both environments. Possibly involves some additional "apt install" parms.

And... I just noticed another slight delta to your above code... You're already running as root -- there's no need to "sudo" the "apt-get" command.
Ahh, good catch! Thanks! I probably don't need to use apt-get right after apt either. I must admit to Copy/Paste without a close review. I don't think I need p7zip or zlib1g-dev, either. But, I will have to check now that they are removed.
EdH is offline   Reply With Quote
Old 2019-10-27, 00:24   #6
Dylan14
 
Dylan14's Avatar
 
"Dylan"
Mar 2017

11·47 Posts
Default

Nice script EdH. But there is one typo: where you have the !make install line for ecm you have an indent, which causes the script to fail. It needs to be inline with the other lines.
Dylan14 is offline   Reply With Quote
Old 2019-10-27, 00:40   #7
EdH
 
EdH's Avatar
 
"Ed Hall"
Dec 2009
Adirondack Mtns

1101001011002 Posts
Default

Quote:
Originally Posted by Dylan14 View Post
Nice script EdH. But there is one typo: where you have the !make install line for ecm you have an indent, which causes the script to fail. It needs to be inline with the other lines.
Thanks! I don't know why those keep showing up. I've already gotten rid of half a dozen.
EdH is offline   Reply With Quote
Old 2019-10-28, 23:35   #8
PhilF
 
PhilF's Avatar
 
Feb 2005
Colorado

523 Posts
Default

This is great! However, I discovered that the number I wanted to work on (M1277) is too large for GPU work.

That sent me on a quest to find suitable numbers that need some ECM thrown at them that were within reach of GPU work, and I found that search difficult.

So where to find some suitable candidates for GPU based ECM curves?

I would like to compare the relative speed of a GPU-enabled GMP-ECM stage 1 with a CPU-based Prime95 stage 1.
PhilF is offline   Reply With Quote
Old 2019-10-29, 02:04   #9
VBCurtis
 
VBCurtis's Avatar
 
"Curtis"
Feb 2005
Riverside, CA

10001000111112 Posts
Default

Quote:
Originally Posted by PhilF View Post

So where to find some suitable candidates for GPU based ECM curves?

I would like to compare the relative speed of a GPU-enabled GMP-ECM stage 1 with a CPU-based Prime95 stage 1.
GPU-ECM is limited to 1019 bits. Prime95 is limited to numbers of a certain form. There's not a ton of overlap between them, but I happen to be working on factorizations of k*2^n-1 (Riesel form).
My next two SNFS jobs are:
Code:
#expr(13*2^881-1)
466877517878596827431051486024691686855414677675172874855133940880576871475985818271788497478759019442908250077983766404430012687428346626398685854186379459430598338289874292283833496457445239
#expr(13*2^892-1)
18506805396968135611155184891404406020831889021055758328923431924399638720432241766053234118351869077918037177534464815730861966812520562472496745230825092848732924799837750268894252467544668175171
I believe these can be run by both programs. They've each had about 2t50 done already, and need something close to t55; so curves at 11e7 or bigger are helpful to my efforts.
If you're curious, you can find project status at mklasson.com/factors
VBCurtis is online now   Reply With Quote
Old 2019-10-29, 02:09   #10
PhilF
 
PhilF's Avatar
 
Feb 2005
Colorado

20B16 Posts
Default

Perfect! That is exactly what I was looking for. I'll do some performance tests and get back with you...
PhilF is offline   Reply With Quote
Old 2019-10-29, 06:19   #11
LaurV
Romulan Interpreter
 
LaurV's Avatar
 
Jun 2011
Thailand

8,837 Posts
Default

Quote:
Originally Posted by EdH View Post
Thanks! I don't know why those keep showing up. I've already gotten rid of half a dozen.
That's the forum, gov'nor!

Since last year when Mike changed I don't know what, I have a continuous battle with random spaces appearing in front of code lines in the code tag (therefore mis-aligning tables or space-sensitive code) or empty lines appearing in between paragraphs. Last time I edited somebody's post to align a table header I had to do it 5 or 6 times, because every time some spaces used to appear in front of a different line after I saved the (correct spaced) post. At the end I was convinced I fixed it, but revisiting the post after few days, some lines inside of the code tag are still mis-aligned. Futile to say, the post still shows I was the last guy to edit it.

Last fiddled with by LaurV on 2019-10-29 at 06:22 Reason: see what I mean? Now I know why Mike made me supermod, to be able to delete empty spaces in old posts.. :P
LaurV is offline   Reply With Quote
Reply

Thread Tools


Similar Threads
Thread Thread Starter Forum Replies Last Post
Google Co-Educational Notebook? Corbeau Cloud Computing 1036 2020-10-22 01:31
Google Colaboratory reference thread kriesel kriesel 15 2020-02-09 17:40
Your browsing session ended unexpectly davieddy Soap Box 1 2010-12-31 20:03
Anyone want to compile an OS X ecm for yoyo? jasong GMP-ECM 1 2009-03-14 11:22
help to compile.... em99010pepe Programming 8 2006-12-06 17:11

All times are UTC. The time now is 05:40.

Thu Oct 22 05:40:47 UTC 2020 up 42 days, 2:51, 0 users, load averages: 1.32, 1.45, 1.47

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.