Compiling Msieve with CUDA support

VictordeHolland · 2018-10-02, 10:52

Hi,

I can compile the latest Msieve using MSYS2/Mingw-64 (GCC7) on Windows without CUDA.

On Linux (Ubuntu 16.04) I can compile Msieve with the standard GCC/make tools.

Can somebody help me learn to compile Msieve using Visual Studio so I can have CUDA support on my Nvidia GTX1080ti?

VictordeHolland · 2018-10-03, 13:00

I've installed:
Visual Studio Community v15
CUDA toolkit 10
And I've downloaded/ extracted the source

How do I proceed?

RichD · 2018-10-03, 15:01

I know nothing about Windows or its environment but GTX 1080ti is a cc 6.1 which needs CUDA 8.0 or higher. Perhaps a lower CUDA version/package would be easier to install ??

VictordeHolland · 2018-10-03, 15:38

Installing CUDA toolkit 8 , 9.2 or 10 should be equally easy.
What I want to learn is the steps I need to do in VS to compile Msieve (with CUDA support).

Do I need to compile GMP6.1.2/MPIR first or can I somehow link to the libs compiled with MSYS2/Mingw64?
Same for GMP-ECM?

ATH · 2018-10-04, 03:12

I was going to post how I compile it with MSYS2, but right now the new svn 1025 fails for me:

Quote:

cd cub && make WIN=1 WIN64=1 sm=300,350,520,610 && cd ..
make[1]: Entering directory 'C:/msys64/home/ATH/msieve/cub'
"/C/CUDA92/dev/bin/nvcc" -gencode=arch=compute_61,code="sm_61,compute_61" -gencode=arch=compute_52,code="sm_52,compute_52" -gencode=arch=compute_35,code="sm_35,compute_35" -gencode=arch=compute_30,code="sm_30,compute_30" -DSM610 -DSM520 -DSM350 -DSM300 -o sort_engine.dll sort_engine.cu -m64 -Xptxas -v -Xcudafe -# -shared -Xptxas -abi=no -I/c/CUDA92/dev/include -I. -O3
c:\msys64\home\ath\msieve\cub\cub\util_arch.cuh(46) : fatal error C1017: invalid integer constant expression
sort_engine.cu
make[1]: *** [Makefile:79: sort_engine.dll] Error 2
make[1]: Leaving directory 'C:/msys64/home/ATH/msieve/cub'
make: *** [Makefile:350: cub/built] Error 2

tServo · 2018-10-04, 15:12

Quote:

Originally Posted by VictordeHolland

Installing CUDA toolkit 8 , 9.2 or 10 should be equally easy.
What I want to learn is the steps I need to do in VS to compile Msieve (with CUDA support).

Do I need to compile GMP6.1.2/MPIR first or can I somehow link to the libs compiled with MSYS2/Mingw64?
Same for GMP-ECM?

I'm 95% sure you would need to compile MPIR first. I've not tried to compile Msieve for many years but have used VS + MPIR on other projects to replace GMP. When compiling MPIR, follow the section that describes GMP compatibility I believe you need to set a compile option to generate the GMP headers.
Also, I'm fairly certain you can't use object files from the MSYS compile, but am not sure.

I don;t know anything about GMP-ECM. That could be a real sticking point if it won't compile using VS. Then you would HAVE to use MSYS/etc to compile it and keep your fingers crossed that its files can be used by VS in the Msieve compile.

VictordeHolland · 2018-10-04, 15:39

I hope that Ben reads this forum, I think he sort of maintains the VS build files?

Gimarel · 2018-10-04, 15:58

Quote:

Originally Posted by VictordeHolland

How do I proceed?

There's a visual studio project under build.cuda.vc15.
That directory contains a readme.txt.
I didn't try to build a windows executable.

Brian Gladman · 2018-10-15, 13:12

Quote:

Originally Posted by Gimarel

There's a visual studio project under build.cuda.vc15.
That directory contains a readme.txt.
I didn't try to build a windows executable.

I am maintaining the Visual Studio 2017 build files for both the non-gpu and gpu versions of msieve.

I have just updated to accommodate the recent source code changes and the two builds now both work for me.

VictordeHolland · 2018-10-26, 14:58

Thanks Brian!

Pthreads is giving me a headache, so I ended up downloading:
pthreads-w32-2-9-1-release
and copied the prebuild .lib and .h and placed them in the msieve/include and msieve/lib folders.

I did a bit of dirty copy/pasting and editing the header files to get VS2017 to compile Msieve (without CUDA) without errors (still got 542 warnings

).

Code:

5>Finished generating code
5>msieve.vcxproj -> C:\FactoringVS\MSIEVE\build.vc15\..\bin\x64\Release\msieve.exe
5>Done building project "msieve.vcxproj".
========== Rebuild All: 5 succeeded, 0 failed, 0 skipped ==========

Unsurprisingly my compiled .exe has issues/errors and or stupid mistakes by me

.

QS on a C70 works with the msieve.exe compiled in VS2017:

Code:

Fri Oct 26 15:48:39 2018  
Fri Oct 26 15:48:39 2018  
Fri Oct 26 15:48:39 2018  Msieve v. 1.54 (SVN unknown)
Fri Oct 26 15:48:39 2018  random seeds: 738aaa48 33b932b2
Fri Oct 26 15:48:39 2018  factoring 3950565477929583959090443657115834369563721138106225828250209513731263 (70 digits)
Fri Oct 26 15:48:40 2018  searching for 15-digit factors
Fri Oct 26 15:48:40 2018  commencing quadratic sieve (70-digit input)
Fri Oct 26 15:48:40 2018  using multiplier of 3
Fri Oct 26 15:48:40 2018  using VC8 32kb sieve core
Fri Oct 26 15:48:40 2018  sieve interval: 12 blocks of size 32768
Fri Oct 26 15:48:40 2018  processing polynomials in batches of 17
Fri Oct 26 15:48:40 2018  using a sieve bound of 218857 (9781 primes)
Fri Oct 26 15:48:40 2018  using large prime bound of 21447986 (24 bits)
Fri Oct 26 15:48:40 2018  using trial factoring cutoff of 24 bits
Fri Oct 26 15:48:40 2018  polynomial 'A' values have 9 factors
Fri Oct 26 15:49:28 2018  10324 relations (4709 full + 5615 combined from 57185 partial), need 9877
Fri Oct 26 15:49:28 2018  begin with 61894 relations
Fri Oct 26 15:49:28 2018  reduce to 15187 relations in 2 passes
Fri Oct 26 15:49:28 2018  attempting to read 15187 relations
Fri Oct 26 15:49:29 2018  recovered 15187 relations
Fri Oct 26 15:49:29 2018  recovered 13240 polynomials
Fri Oct 26 15:49:29 2018  attempting to build 10324 cycles
Fri Oct 26 15:49:29 2018  found 10324 cycles in 1 passes
Fri Oct 26 15:49:29 2018  distribution of cycle lengths:
Fri Oct 26 15:49:29 2018     length 1 : 4709
Fri Oct 26 15:49:29 2018     length 2 : 5615
Fri Oct 26 15:49:29 2018  largest cycle: 2 relations
Fri Oct 26 15:49:29 2018  matrix is 9781 x 10324 (1.5 MB) with weight 299988 (29.06/col)
Fri Oct 26 15:49:29 2018  sparse part has weight 299988 (29.06/col)
Fri Oct 26 15:49:29 2018  filtering completed in 3 passes
Fri Oct 26 15:49:29 2018  matrix is 7990 x 8054 (1.2 MB) with weight 246625 (30.62/col)
Fri Oct 26 15:49:29 2018  sparse part has weight 246625 (30.62/col)
Fri Oct 26 15:49:29 2018  commencing Lanczos iteration
Fri Oct 26 15:49:29 2018  memory use: 1.6 MB
Fri Oct 26 15:49:29 2018  lanczos halted after 128 iterations (dim = 7984)
Fri Oct 26 15:49:29 2018  recovered 60 nontrivial dependencies
Fri Oct 26 15:49:29 2018  p35 factor: 33637310674071348724927955857253537
Fri Oct 26 15:49:29 2018  p36 factor: 117445937227520353139789517076610399
Fri Oct 26 15:49:29 2018  elapsed time 00:00:50

But RSA100 fails filtering

Code:

Fri Oct 26 16:38:42 2018  commencing relation filtering
Fri Oct 26 16:38:42 2018  estimated available RAM is 32715.2 MB
Fri Oct 26 16:38:42 2018  commencing duplicate removal, pass 1
Fri Oct 26 16:39:29 2018  found 356670 hash collisions in 4361977 relations
Fri Oct 26 16:39:29 2018  commencing duplicate removal, pass 2
Fri Oct 26 16:39:30 2018  found 255002 duplicates and 4106975 unique relations
Fri Oct 26 16:39:30 2018  memory use: 16.3 MB
Fri Oct 26 16:39:30 2018  reading ideals above 100000
Fri Oct 26 16:39:30 2018  commencing singleton removal, initial pass
Fri Oct 26 16:40:05 2018  memory use: 0.1 MB
Fri Oct 26 16:40:05 2018  reading all ideals from disk
Fri Oct 26 16:40:05 2018  memory use: 31.3 MB
Fri Oct 26 16:40:05 2018  commencing in-memory singleton removal
Fri Oct 26 16:40:05 2018  begin with 4106975 relations and 13 unique ideals
Fri Oct 26 16:40:05 2018  reduce to 4106968 relations and 1 ideals in 2 passes
Fri Oct 26 16:40:05 2018  max relations containing the same ideal: 2
Fri Oct 26 16:40:05 2018  relations with 0 large ideals: 4106966
Fri Oct 26 16:40:05 2018  relations with 1 large ideals: 2
Fri Oct 26 16:40:05 2018  relations with 2 large ideals: 0
Fri Oct 26 16:40:05 2018  relations with 3 large ideals: 0
Fri Oct 26 16:40:05 2018  relations with 4 large ideals: 0
Fri Oct 26 16:40:05 2018  relations with 5 large ideals: 0
Fri Oct 26 16:40:05 2018  relations with 6 large ideals: 0
Fri Oct 26 16:40:05 2018  relations with 7+ large ideals: 0
Fri Oct 26 16:40:05 2018  commencing 2-way merge
Fri Oct 26 16:40:05 2018  reduce to 4106967 relation sets and 0 unique ideals
Fri Oct 26 16:40:05 2018  commencing full merge
Fri Oct 26 16:40:05 2018  memory use: 78.3 MB
Fri Oct 26 16:40:06 2018  found 4106967 cycles, need 19414
 Fri Oct 26 16:40:06 2018  weight of 19414 cycles is about 0 (0.00/cycle)

(the same relations do work with Msieve SVN1026 compiled with MSYS2/Mingw64)

Brian Gladman · 2018-10-27, 12:54

That's a bit of a pain :-(

I have just updated my VS2017 pthreads build at: https://github.com/BrianGladman/pthreads if that is any help. I have also updated the VS2017 msieve gpu build to CUDA 10 (https://svn.code.sf.net/p/msieve/code/trunk)

It might be worth trying this since I had to hack NVIDIA CUDA 9 a bit to get it to work (CUDA 10 builds without any issues).

2018-10-02, 10:52	#1
VictordeHolland "Victor de Hollander" Aug 2011 the Netherlands 2³×3×7² Posts	Compiling Msieve with CUDA support Hi, I can compile the latest Msieve using MSYS2/Mingw-64 (GCC7) on Windows without CUDA. On Linux (Ubuntu 16.04) I can compile Msieve with the standard GCC/make tools. Can somebody help me learn to compile Msieve using Visual Studio so I can have CUDA support on my Nvidia GTX1080ti?

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Compiling Msieve with GPU support	LegionMammal978	Msieve	6	2017-02-09 04:28
AMD To Support CUDA	Smokingenius	GPU Computing	9	2015-11-18 23:36
Compiling CUDA programs on VS 2010 Express	ET_	GPU Computing	9	2011-03-24 10:11
Msieve with GPU support	jasonp	Msieve	223	2011-03-11 19:30
Msieve with GNFS support	R.D. Silverman	Msieve	465	2010-01-11 20:59

2018-10-03, 13:00	#2
VictordeHolland "Victor de Hollander" Aug 2011 the Netherlands 2³·3·7² Posts	I've installed: Visual Studio Community v15 CUDA toolkit 10 And I've downloaded/ extracted the source How do I proceed?

2018-10-03, 15:01	#3
RichD Sep 2008 Kansas 41×79 Posts	I know nothing about Windows or its environment but GTX 1080ti is a cc 6.1 which needs CUDA 8.0 or higher. Perhaps a lower CUDA version/package would be easier to install ??

2018-10-03, 15:38	#4
VictordeHolland "Victor de Hollander" Aug 2011 the Netherlands 2³×3×7² Posts	Installing CUDA toolkit 8 , 9.2 or 10 should be equally easy. What I want to learn is the steps I need to do in VS to compile Msieve (with CUDA support). Do I need to compile GMP6.1.2/MPIR first or can I somehow link to the libs compiled with MSYS2/Mingw64? Same for GMP-ECM?

2018-10-04, 15:39	#7
VictordeHolland "Victor de Hollander" Aug 2011 the Netherlands 498₁₆ Posts	I hope that Ben reads this forum, I think he sort of maintains the VS build files?

2018-10-27, 12:54	#11
Brian Gladman May 2008 Worcester, United Kingdom 20B₁₆ Posts	That's a bit of a pain :-( I have just updated my VS2017 pthreads build at: https://github.com/BrianGladman/pthreads if that is any help. I have also updated the VS2017 msieve gpu build to CUDA 10 (https://svn.code.sf.net/p/msieve/code/trunk) It might be worth trying this since I had to hack NVIDIA CUDA 9 a bit to get it to work (CUDA 10 builds without any issues).