Compiling YAFU for Windows using mingw

bsquared · 2017-04-22, 01:36

Here is as close as I can get to a recipe for compiling your own copy of YAFU for windows. I have done this for Window 10 and versions of the tools available as of this post date.

Note that you can also build YAFU using the Microsoft Visual Studio environment - I will start a separate thread on that path.

Step 1: Install mingw-x64 and msys
These are the main tools that get you close to a linux environment on windows.
This is a good guide.
The basics steps are:
1) Go to http://sourceforge.net/projects/mingw-w64/ download the installer and run it. Configure it for x86_64 architecture, win32 threading, and seh exceptions. Install to C:\mingw-w64 (or similar)
2) Go to http://sourceforge.net/projects/ming...inary-packages download the mingw-builds-install.exe and install the latest version to c:\ to create c:\msys
There is a msys package manager where you can select libraries/packages to install. I chose most of the ones dealing with compression and tcl/tk; they can come in handy. You won't need gmp, as we will compile/install the latest version later.
3) create an empty c:\msys\mingw folder
4) Edit C:\msys\etc\fstab to add the line C:/MinGW-W64/mingw64 /mingw
and optionally a path to mount your homedirectory
5) download the latest 64-bit yasm and save it as c:\msys\bin\yasm.exe
6) Other optional stuff - see Ross's guide

Step 2: create a projects folder
Somewhere in your home directory on the C drive (that you have hopefully mounted in /etc/fstab so you can navigate to it in an msys terminal), create a top level directory where all necessary dependencies will go. I call mine /projects/factoring.

Create the following empty directories:
projects/factoring/gmp
projects/factoring/ecm
projects/factoring/msieve
projects/factoring/yafu

Step 3: install gmp
1) Go to https://gmplib.org/ and download the latest source tarball.
2) Put it in your gmp/ folder.
3) Unzip it. (This was almost the most time consuming part for me, as it took me a while to find something that would decompress the .lz file. Eventually http://download.savannah.gnu.org/releases/lzip/ worked.)
In msys, extract the archive with e.g., the command tar -xvf gmp-6.1.2.tar
4) Navigate to the newly created gmp/gmp-6.1.2 folder and run the command ./configure --prefix=/path/to/your/gmp/folder/install-6.1.2
The prefix argument specifies where all of the installed libraries will go. The default I think is /usr/lib, which may or may not exist in the mounted filesystem.
My exact command was ./configure --prefix=/home/Ben/projects/factoring/gmp/install-6.1.2
You can find out what your current path is by running the command 'pwd' in the msys terminal.
5) Run make
6) Run make install

Step 4: install ecm
1) Go to https://gforge.inria.fr/frs/?group_i...elease_id=9946 and download the latest source tarball. For me it was 7.0.4
2) Put it in your ecm/ folder.
3) Run the command tar -xvzf ecm-7.0.4.tar.gz
4) Go into the new directory and run ./configure --prefix=/path/to/your/ecm/folder/install-6.1.2 --with-gmp=/path/to/your/gmp/folder/install-6.1.2
This both tells it where to put new ecm libraries and where to find your gmp libraries.
5) run make
6) run make install

Step 5: install msieve
1) Go to https://sourceforge.net/p/msieve/code/HEAD/tree/trunk/ and click download snapshot to get the latest msieve source
2) I had to edit the Makefile to point to the gmp and ecm libraries.
Find and change lines to make them look like this:
CFLAGS = $(OPT_FLAGS) $(MACHINE_FLAGS) $(WARN_FLAGS) \
-DMSIEVE_SVN_VERSION=""$(SVN_VERSION)"" \
-I. -Iaprcl -Iinclude -Ignfs -Ignfs/poly -Ignfs/poly/stage1 \
-I../gmp/install-6.1.2/include/ -I../ecm/install-7.0.4/include/

ifeq ($(ECM),1)
CFLAGS += -DHAVE_GMP_ECM
LIBS += -L../gmp/install-6.1.2/lib -L../ecm/install-7.0.4/lib -lecm
endif
3) Navigate to the top level msieve/ directory in msys and run 'make all ECM=1 NO_ZLIB=1 '
If you want CUDA that's your business :) Maybe we can sort out how to get msieve-cuda working with yafu in this thread. Right now it isn't supported.

Step 6: install yafu
1) Go to https://sourceforge.net/p/yafu/code/...e/branches/wip and click download snapshot to get the latest yafu source
2) Edit the Makefile.mingw to point to the gmp and ecm libraries.
Find and change lines to make them look like this:
INC += -I../gmp/install-6.1.2/include/
LIBS += -L../gmp/install-6.1.2/lib/

INC += -I../ecm/install-7.0.4/include
LIBS += -L../ecm/install-7.0.4/lib

3) run 'make -f Makefile.mingw x86_64 NFS=1 USE_SSE41=1'
or USE_AVX2 if your computer supports it. or omit both if it supports neither.
4) get the 64-bit ggnfs windows binaries and put them somewhere next to the other tools, e.g., /project/factoring/ggnfs
5) edit yafu.ini to point to the ggnfs directory and to the ecm binary created in Step 2.

Hopefully all of that worked and you now have a nice factoring environment... if not feel free to ask questions here.

Happy factoring!

Stargate38 · 2017-04-24, 22:09

When I get to Step 1, Part 2 (running mingw-builds-install.exe), I get the following error message:

Code:

Cannot download repository.txt [0]

How do I get it to successfully download "repository.txt"?

bsquared · 2017-04-24, 22:30

I got that message too, and I can't remember how I got around it. I may have just tried again later and had it work.

Or, searching I found this thread:
https://sourceforge.net/p/mingw-w64/...sage/32984065/

which refers to this thread
https://sourceforge.net/p/mingw-w64/...sage/32967954/

whence you can download the 64-bit win32-seh toolchain directly.

Stargate38 · 2017-04-24, 23:11

I'm talking about the step after that (http://sourceforge.net/projects/ming...inary-packages).

bsquared · 2017-04-24, 23:57

Quote:

Originally Posted by Stargate38

I'm talking about the step after that (http://sourceforge.net/projects/ming...inary-packages).

You could try getting one of the pre-packaged builds in the list of "msys+7za+wget+svn+git+" type links on that page. Extract to e.g., C:\msys

skan · 2017-04-27, 23:10

Could you upload the resulting binaries, please?

skan · 2017-05-09, 14:45

We just also need a newer version of ggnfs or being able to compile it.

Jayder · 2017-10-07, 00:56

I've run into issues at step 5, installing msieve. I've edited the Makefile as instructed, and I believe everything else is in the right place. Hopefully I am not revealing any sensitive information here:

Code:

$ make all ECM=1 NO_ZLIB=1
rm -f libmsieve.a
ar r libmsieve.a aprcl/mpz_aprcl32.o common/filter/clique.o common/filter/filter.o common/filter/merge.o common/filter/merge_post.o common/filter/merge_pre.o common/filter/merge_util.o common/filter/singleton.o common/lanczos/lanczos.o common/lanczos/lanczos_io.o common/lanczos/lanczos_matmul0.o common/lanczos/lanczos_matmul1.o common/lanczos/lanczos_matmul2.o common/lanczos/lanczos_pre.o common/lanczos/lanczos_vv.o common/lanczos/matmul_util.o common/smallfact/gmp_ecm.o common/smallfact/smallfact.o common/smallfact/squfof.o common/smallfact/tinyqs.o common/batch_factor.o common/cuda_xface.o common/dickman.o common/driver.o common/expr_eval.o common/hashtable.o common/integrate.o common/minimize.o common/minimize_global.o common/mp.o common/polyroot.o common/prime_delta.o common/prime_sieve.o common/savefile.o common/strtoll.o common/thread.o common/util.o mpqs/gf2.qo  mpqs/mpqs.qo  mpqs/poly.qo  mpqs/relation.qo  mpqs/sieve.qo  mpqs/sqrt.qo  mpqs/sieve_core_generic_32k.qo  mpqs/sieve_core_generic_64k.qo gnfs/poly/poly.no gnfs/poly/poly_param.no gnfs/poly/poly_skew.no gnfs/poly/polyutil.no gnfs/poly/root_score.no gnfs/poly/size_score.no gnfs/poly/stage1/stage1.no gnfs/poly/stage1/stage1_roots.no gnfs/poly/stage2/optimize.no gnfs/poly/stage2/optimize_deg6.no gnfs/poly/stage2/root_sieve.no gnfs/poly/stage2/root_sieve_deg45_x.no gnfs/poly/stage2/root_sieve_deg5_xy.no gnfs/poly/stage2/root_sieve_deg6_x.no gnfs/poly/stage2/root_sieve_deg6_xy.no gnfs/poly/stage2/root_sieve_deg6_xyz.no gnfs/poly/stage2/root_sieve_line.no gnfs/poly/stage2/root_sieve_util.no gnfs/poly/stage2/stage2.no gnfs/filter/duplicate.no gnfs/filter/filter.no gnfs/filter/singleton.no gnfs/sieve/sieve_line.no gnfs/sieve/sieve_util.no gnfs/sqrt/sqrt.no gnfs/sqrt/sqrt_a.no gnfs/fb.no gnfs/ffpoly.no gnfs/gf2.no gnfs/gnfs.no gnfs/relation.no gnfs/poly/stage1/stage1_sieve_cpu.no gnfs/poly/stage1/stage1_sieve_cpu.no
C:\MinGW-W64\mingw64\bin\ar.exe: creating libmsieve.a
ranlib libmsieve.a
gcc -O3 -fomit-frame-pointer -march=native  -D_FILE_OFFSET_BITS=64 -DNDEBUG -D_LARGEFILE64_SOURCE  -Wall -W  -DMSIEVE_SVN_VERSION=""exported""  -I. -Iaprcl -Iinclude -Ignfs -Ignfs/poly -Ignfs/poly/stage1  -I../gmp/install-6.1.2/include/ -I../ecm/install-7.0.4/include/ -DHAVE_GMP_ECM -DNO_ZLIB demo.c -o msieve  \
                libmsieve.a -L../gmp/install-6.1.2/lib -L../ecm/install-7.0.4/lib -lecm -ldl -lgmp -lm -lpthread
demo.c: In function 'print_usage':
<command-line>:0:20: error: 'exported' undeclared (first use in this function)
demo.c:86:6: note: in expansion of macro 'MSIEVE_SVN_VERSION'
      MSIEVE_SVN_VERSION);
      ^~~~~~~~~~~~~~~~~~
<command-line>:0:20: note: each undeclared identifier is reported only once for each function it appears in
demo.c:86:6: note: in expansion of macro 'MSIEVE_SVN_VERSION'
      MSIEVE_SVN_VERSION);
      ^~~~~~~~~~~~~~~~~~
demo.c: In function 'main':
demo.c:448:13: warning: this statement may fall through [-Wimplicit-fallthrough=]
       flags |= MSIEVE_FLAG_NFS_LA_RESTART;
       ~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
demo.c:449:6: note: here
      case '2': /* fall through */
      ^~~~
Makefile:276: recipe for target `all' failed
make: *** [all] Error 1

jasonp · 2017-10-08, 12:24

Sounds like the 'svnversion' command in the makefile didn't run, or failed. Even if you didn't use subversion to check out the source, there should have been a fallback path for that.

Jayder · 2017-10-09, 05:07

I noticed an error in the following, but I think I had already accounted for it. I'll mention it now, though:

Quote:

Originally Posted by bsquared

4) Go into the new directory and run ./configure --prefix=/path/to/your/ecm/folder/install-6.1.2 --with-gmp=/path/to/your/gmp/folder/install-6.1.2

The ecm install folder should be, 'install-7.0.4'.

I deleted my msieve folder and started that step from scratch, and I ran into different problems. I then tried again starting from step 2, and received the same second problem. Could it be a problem with msys or mingw if gmp and gmp-ecm both compile successfully? Is it an issue with me, or with msieve/yafu?

Here is my full log for steps 5 and 6: https://pastebin.com/BKeAmTmi (You may notice that I kept the ecm folder as 6.1.2 but it is properly referenced)

There is a bit of reading between the lines in this guide. I was able to fill in a few of the blanks, but perhaps I'm missing something that isn't explicitly stated, or isn't easily searchable.

Thanks for your time, jasonp.

Jayder · 2017-10-19, 01:15

I've reverted to yafu 1.33 since so much time is wasted on poly select due to the bug with 1.34. If anybody would be so kind as to help me with the problems outlined in my post above, I'd be very grateful. Could someone perhaps try following the instructions themselves to see if it is an error on my side or if it is something else? It's curious that no one in this thread has posted to say they have successfully compiled it. Of course msieve is compilable, but it would seem that we're missing something.

I know you guys are busy, and I don't want to be one of those nags. Whether or not you have time to help, thank you.

2017-04-22, 01:36	#1
bsquared "Ben" Feb 2007 3,733 Posts	Compiling YAFU for Windows using mingw Here is as close as I can get to a recipe for compiling your own copy of YAFU for windows. I have done this for Window 10 and versions of the tools available as of this post date. Note that you can also build YAFU using the Microsoft Visual Studio environment - I will start a separate thread on that path. Step 1: Install mingw-x64 and msys These are the main tools that get you close to a linux environment on windows. This is a good guide. The basics steps are: 1) Go to http://sourceforge.net/projects/mingw-w64/ download the installer and run it. Configure it for x86_64 architecture, win32 threading, and seh exceptions. Install to C:\mingw-w64 (or similar) 2) Go to http://sourceforge.net/projects/ming...inary-packages download the mingw-builds-install.exe and install the latest version to c:\ to create c:\msys There is a msys package manager where you can select libraries/packages to install. I chose most of the ones dealing with compression and tcl/tk; they can come in handy. You won't need gmp, as we will compile/install the latest version later. 3) create an empty c:\msys\mingw folder 4) Edit C:\msys\etc\fstab to add the line C:/MinGW-W64/mingw64 /mingw and optionally a path to mount your homedirectory 5) download the latest 64-bit yasm and save it as c:\msys\bin\yasm.exe 6) Other optional stuff - see Ross's guide Step 2: create a projects folder Somewhere in your home directory on the C drive (that you have hopefully mounted in /etc/fstab so you can navigate to it in an msys terminal), create a top level directory where all necessary dependencies will go. I call mine /projects/factoring. Create the following empty directories: projects/factoring/gmp projects/factoring/ecm projects/factoring/msieve projects/factoring/yafu Step 3: install gmp 1) Go to https://gmplib.org/ and download the latest source tarball. 2) Put it in your gmp/ folder. 3) Unzip it. (This was almost the most time consuming part for me, as it took me a while to find something that would decompress the .lz file. Eventually http://download.savannah.gnu.org/releases/lzip/ worked.) In msys, extract the archive with e.g., the command tar -xvf gmp-6.1.2.tar 4) Navigate to the newly created gmp/gmp-6.1.2 folder and run the command ./configure --prefix=/path/to/your/gmp/folder/install-6.1.2 The prefix argument specifies where all of the installed libraries will go. The default I think is /usr/lib, which may or may not exist in the mounted filesystem. My exact command was ./configure --prefix=/home/Ben/projects/factoring/gmp/install-6.1.2 You can find out what your current path is by running the command 'pwd' in the msys terminal. 5) Run make 6) Run make install Step 4: install ecm 1) Go to https://gforge.inria.fr/frs/?group_i...elease_id=9946 and download the latest source tarball. For me it was 7.0.4 2) Put it in your ecm/ folder. 3) Run the command tar -xvzf ecm-7.0.4.tar.gz 4) Go into the new directory and run ./configure --prefix=/path/to/your/ecm/folder/install-6.1.2 --with-gmp=/path/to/your/gmp/folder/install-6.1.2 This both tells it where to put new ecm libraries and where to find your gmp libraries. 5) run make 6) run make install Step 5: install msieve 1) Go to https://sourceforge.net/p/msieve/code/HEAD/tree/trunk/ and click download snapshot to get the latest msieve source 2) I had to edit the Makefile to point to the gmp and ecm libraries. Find and change lines to make them look like this: CFLAGS = $(OPT_FLAGS) $(MACHINE_FLAGS) $(WARN_FLAGS) \ -DMSIEVE_SVN_VERSION=""$(SVN_VERSION)"" \ -I. -Iaprcl -Iinclude -Ignfs -Ignfs/poly -Ignfs/poly/stage1 \ -I../gmp/install-6.1.2/include/ -I../ecm/install-7.0.4/include/ ifeq ($(ECM),1) CFLAGS += -DHAVE_GMP_ECM LIBS += -L../gmp/install-6.1.2/lib -L../ecm/install-7.0.4/lib -lecm endif 3) Navigate to the top level msieve/ directory in msys and run 'make all ECM=1 NO_ZLIB=1 ' If you want CUDA that's your business :) Maybe we can sort out how to get msieve-cuda working with yafu in this thread. Right now it isn't supported. Step 6: install yafu 1) Go to https://sourceforge.net/p/yafu/code/...e/branches/wip and click download snapshot to get the latest yafu source 2) Edit the Makefile.mingw to point to the gmp and ecm libraries. Find and change lines to make them look like this: INC += -I../gmp/install-6.1.2/include/ LIBS += -L../gmp/install-6.1.2/lib/ INC += -I../ecm/install-7.0.4/include LIBS += -L../ecm/install-7.0.4/lib 3) run 'make -f Makefile.mingw x86_64 NFS=1 USE_SSE41=1' or USE_AVX2 if your computer supports it. or omit both if it supports neither. 4) get the 64-bit ggnfs windows binaries and put them somewhere next to the other tools, e.g., /project/factoring/ggnfs 5) edit yafu.ini to point to the ggnfs directory and to the ecm binary created in Step 2. Hopefully all of that worked and you now have a nice factoring environment... if not feel free to ask questions here. Happy factoring! Last fiddled with by bsquared on 2017-04-25 at 01:47 Reason: cleanup

2017-04-24, 22:09	#2
Stargate38 "Daniel Jackson" May 2011 14285714285714285714 3×251 Posts	When I get to Step 1, Part 2 (running mingw-builds-install.exe), I get the following error message: Code: Cannot download repository.txt [0] How do I get it to successfully download "repository.txt"?

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Guide to compiling YAFU	Mr. Odd	YAFU	4	2017-04-24 15:40
Compiling YAFU under MinGW	wombatman	YAFU	10	2016-01-21 19:48
Need help compiling YAFU from SVN.	Stargate38	YAFU	14	2016-01-20 21:46
compiling svn 427 for Windows 64 bit.	skan	NFSNET Discussion	7	2012-04-18 10:30
Compiling ECM 5.0.3 for windows	BotXXX	Factoring	25	2005-09-13 12:24

2017-04-24, 22:30	#3
bsquared "Ben" Feb 2007 3,733 Posts	I got that message too, and I can't remember how I got around it. I may have just tried again later and had it work. Or, searching I found this thread: https://sourceforge.net/p/mingw-w64/...sage/32984065/ which refers to this thread https://sourceforge.net/p/mingw-w64/...sage/32967954/ whence you can download the 64-bit win32-seh toolchain directly.

2017-04-24, 23:11	#4
Stargate38 "Daniel Jackson" May 2011 14285714285714285714 1011110001₂ Posts	I'm talking about the step after that (http://sourceforge.net/projects/ming...inary-packages).

2017-04-27, 23:10	#6
skan Apr 2012 2×47 Posts	Could you upload the resulting binaries, please?

2017-05-09, 14:45	#7
skan Apr 2012 94₁₀ Posts	We just also need a newer version of ggnfs or being able to compile it.

2017-10-08, 12:24	#9
jasonp Tribal Bullet Oct 2004 3²·5·79 Posts	Sounds like the 'svnversion' command in the makefile didn't run, or failed. Even if you didn't use subversion to check out the source, there should have been a fallback path for that.

2017-10-19, 01:15	#11
Jayder Dec 2012 3²·31 Posts	I've reverted to yafu 1.33 since so much time is wasted on poly select due to the bug with 1.34. If anybody would be so kind as to help me with the problems outlined in my post above, I'd be very grateful. Could someone perhaps try following the instructions themselves to see if it is an error on my side or if it is something else? It's curious that no one in this thread has posted to say they have successfully compiled it. Of course msieve is compilable, but it would seem that we're missing something. I know you guys are busy, and I don't want to be one of those nags. Whether or not you have time to help, thank you.