Building yafu on windows/linux

[2147483647]

After trying and failing several times to build yafu, I just found this. When running "make all" on windows 10, I get the following output:

The system cannot find the path specified.
make -C /gmp-6.0.0 uninstall clean
The system cannot find the path specified.
The system cannot find the path specified.
The system cannot find the path specified.
The system cannot find the path specified.
The system cannot find the path specified.
The system cannot find the path specified.
The system cannot find the path specified.
The system cannot find the path specified.
The system cannot find the path specified.
The system cannot find the path specified.
The system cannot find the path specified.
make[1]: *** /gmp-6.0.0: No such file or directory.  Stop.
Makefile:33: recipe for target 'clean_gmp' failed
make: [clean_gmp] Error 2 (ignored)
The system cannot find the path specified.
make -C /ecm-6.4.4 uninstall clean
The system cannot find the path specified.
The system cannot find the path specified.
The system cannot find the path specified.
The system cannot find the path specified.
The system cannot find the path specified.
The system cannot find the path specified.
The system cannot find the path specified.
The system cannot find the path specified.
The system cannot find the path specified.
The system cannot find the path specified.
The system cannot find the path specified.
make[1]: *** /ecm-6.4.4: No such file or directory.  Stop.
Makefile:36: recipe for target 'clean_ecm' failed
make: [clean_ecm] Error 2 (ignored)
The system cannot find the path specified.
make -C /msieve-1.52 clean
The system cannot find the path specified.
The system cannot find the path specified.
The system cannot find the path specified.
The system cannot find the path specified.
The system cannot find the path specified.
The system cannot find the path specified.
The system cannot find the path specified.
The system cannot find the path specified.
The system cannot find the path specified.
The system cannot find the path specified.
The system cannot find the path specified.
make[1]: *** /msieve-1.52: No such file or directory.  Stop.
Makefile:39: recipe for target 'clean_msieve' failed
make: *** [clean_msieve] Error 2

and on linux, it works correctly for a bit, until it gets to ecm configure, then I get this:

...
checking for malloc_usable_size... yes
checking gmp.h usability... yes
checking gmp.h presence... yes
checking for gmp.h... yes
checking for recent GMP... no
configure: error: GMP 5.0.0 or newer is required
make: *** [ecm] Error 1

even though the download contains gmp 6.0.0.

There are no spaces in any file paths, so that's not the problem. Does anyone know how to get it to build properly?

[Dubslow]

Looks like the package you're using has a bug: the path in question, at least on Linux, should be ./gmp-6.0.0, not /gmp-6.0.0. On Linux at least, the latter form is an *absolute* path, where the author meant it to be relative to the current directory ..

Edit: Reviewing the source in question, it looks like this line is the problem. For whatever reason the command fails, leaving $PKG_DIR empty, and thus rendering the variables derived from it, such as:
GMP_DIR=$(PKG_DIR)/gmp-6.0.0
ECM_DIR=$(PKG_DIR)/ecm-6.4.4
MSIEVE_DIR=$(PKG_DIR)/msieve-1.52
GGNFS_DIR=$(PKG_DIR)/ggnfs
YAFU_DIR=$(PKG_DIR)/yafu-1.34.3

absolute paths instead of relative by mistake.

This category of bug is reasonably well understood in the bash world, and can have potentially disastrous comments if the generated paths are passed to e.g. an rm call instead of a make call.

I cannot say how the failing command could be fixed, but it would be a good idea anyways to error check that command for the reason above.

In your case, you can probably work around it by manually replacing line 9 of the script with something likePKG_DIR=/absolute/path/to/your/copy/of/the/package instead of relying on the faulty command.

[2147483647]

Originally Posted by Dubslow

In your case, you can probably work around it by manually replacing line 9 of the script with something likePKG_DIR=/absolute/path/to/your/copy/of/the/package instead of relying on the faulty command.

I've changed this now and the first 2 stages work (make clean and make populate_prefix) but still nothing else. The next issue seems to happen when running make lasieve_binaries, where I get this:

mkdir C:/Users/benwh/Downloads/YAFU_SETUP_PACKAGE/ggnfs/bin
mkdir: cannot create directory `C:/Users/benwh/Downloads/YAFU_SETUP_PACKAGE/ggnfs/bin': File exists
Makefile:71: recipe for target 'lasieve_binaries' failed
make: [lasieve_binaries] Error 1 (ignored)
cp C:/Users/benwh/Downloads/YAFU_SETUP_PACKAGE/lasieve_bin/gnfs-lasieve4I1* C:/Users/benwh/Downloads/YAFU_SETUP_PACKAGE/ggnfs/bin
chmod +x C:/Users/benwh/Downloads/YAFU_SETUP_PACKAGE/ggnfs/bin/gnfs-lasieve4I1*
touch C:/Users/benwh/Downloads/YAFU_SETUP_PACKAGE/ggnfs/bin/gnfs-lasieve4I1*

Still not sure what I'd need to change on linux though, the directory names don't appear to be a problem. It finds the gmp.h header file correctly, but doesn't recognise the version number for some reason.

[Dubslow]

make: [lasieve_binaries] Error 1 (ignored)

Are you sure it didn't work? What's in your C:/Users/benwh/Downloads/YAFU_SETUP_PACKAGE/ggnfs/bin directory?

Not immediately sure how to proceed with diagnosis of the linux gmp issue. Can you find, in the make output, what the precise ./configure command is?

[2147483647]

The lasieve binaries are in the bin folder, but there's no output from the compiler and the command runs very quickly, making me think that it just copied the binaries from YAFU_SETUP_PACKAGE/lasieve_bin (the files are identical). On linux, the ggnfs files are actually compiled properly. As for the rest of the output of "make ggnfs", it's showing some very weird directory names:

cd C:/Users/benwh/Downloads/YAFU_SETUP_PACKAGE/ggnfs && \
make  -C C:/Users/benwh/Downloads/YAFU_SETUP_PACKAGE/ggnfs x86_64
make[1]: Entering directory 'C:/Users/benwh/Downloads/YAFU_SETUP_PACKAGE/ggnfs'
make[2]: Entering directory 'C:/Users/benwh/Downloads/YAFU_SETUP_PACKAGE/ggnfs'
echo "#define GGNFS_VERSION \"0.77.1-20060722-k8\"" > include/version.h
make[3]: Entering directory 'C:/Users/benwh/Downloads/YAFU_SETUP_PACKAGE/ggnfs/src'
gcc -I. -I.. -I../include -IC:\MinGW\msys\1.0\local\include -IC;C:\MinGW\msys\1.0\Users\benwh\Downloads\YAFU_SETUP_PACKAGE\prefix\include -DNDEBUG -O3 -funroll-loops -finline-functions -ftracer -fomit-frame-pointer -W -Wall -march=k8 -pipe -DGGNFS_HOST_GENERIC -DMALLOC_REPORTING -DGMP_BUG -o getprimes.o -c getprimes.c
gcc.exe: fatal error: no input files
compilation terminated.
/usr/bin/sh: C:MinGWmsys1.0UsersbenwhDownloadsYAFU_SETUP_PACKAGEprefixinclude: command not found
Makefile:91: recipe for target 'getprimes.o' failed
make[3]: *** [getprimes.o] Error 127
make[3]: Leaving directory 'C:/Users/benwh/Downloads/YAFU_SETUP_PACKAGE/ggnfs/src'
Makefile:88: recipe for target 'common' failed
make[2]: *** [common] Error 2
make[2]: Leaving directory 'C:/Users/benwh/Downloads/YAFU_SETUP_PACKAGE/ggnfs'
Makefile:61: recipe for target 'x86_64' failed
make[1]: *** [x86_64] Error 2
make[1]: Leaving directory 'C:/Users/benwh/Downloads/YAFU_SETUP_PACKAGE/ggnfs'
Makefile:77: recipe for target 'ggnfs' failed
make: *** [ggnfs] Error 2

On linux, after running make clean, make populate_prefix, make ggnfs, make gmp, cd , this is the complete output from running ./configure --prefix=/home/b/bw113/Downloads/YAFU_SETUP_PACKAGE/prefix

checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for a sed that does not truncate output... /bin/sed
checking for CC and CFLAGS in gmp.h... yes CC=gcc -std=gnu99 CFLAGS=-O2 -pedantic -m64 -mtune=k8
checking whether CC=gcc -std=gnu99 and CFLAGS=-O2 -pedantic -m64 -mtune=k8 works... yes
checking for gcc... gcc -std=gnu99
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc -std=gnu99 accepts -g... yes
checking for gcc -std=gnu99 option to accept ISO C89... none needed
checking for style of include used by make... GNU
checking dependency style of gcc -std=gnu99... gcc3
checking dependency style of gcc -std=gnu99... (cached) gcc3
checking dependency style of gcc -std=gnu99... gcc3
checking whether gcc -std=gnu99 and cc understand -c and -o together... yes
checking how to print strings... printf
checking for a sed that does not truncate output... (cached) /bin/sed
checking for fgrep... /bin/grep -F
checking for ld used by gcc -std=gnu99... /usr/bin/ld
checking if the linker (/usr/bin/ld) is GNU ld... yes
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
checking the name lister (/usr/bin/nm -B) interface... BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 3458764513820540925
checking whether the shell understands some XSI constructs... yes
checking whether the shell understands "+="... yes
checking how to convert x86_64-unknown-linux-gnu file names to x86_64-unknown-linux-gnu format... func_convert_file_noop
checking how to convert x86_64-unknown-linux-gnu file names to toolchain format... func_convert_file_noop
checking for /usr/bin/ld option to reload object files... -r
checking for objdump... objdump
checking how to recognize dependent libraries... pass_all
checking for dlltool... no
checking how to associate runtime and link libraries... printf %s\n
checking for ar... ar
checking for archiver @FILE support... @
checking for strip... strip
checking for ranlib... ranlib
checking command to parse /usr/bin/nm -B output from gcc -std=gnu99 object... ok
checking for sysroot... no
checking for mt... no
checking if : is a manifest tool... no
checking how to run the C preprocessor... gcc -std=gnu99 -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for dlfcn.h... yes
checking for objdir... .libs
checking if gcc -std=gnu99 supports -fno-rtti -fno-exceptions... no
checking for gcc -std=gnu99 option to produce PIC... -fPIC -DPIC
checking if gcc -std=gnu99 PIC flag -fPIC -DPIC works... yes
checking if gcc -std=gnu99 static flag -static works... yes
checking if gcc -std=gnu99 supports -c -o file.o... yes
checking if gcc -std=gnu99 supports -c -o file.o... (cached) yes
checking whether the gcc -std=gnu99 linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking dynamic linker characteristics... GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... no
checking whether to build static libraries... yes
checking for int64_t... yes
checking for uint64_t... yes
checking for long long int... yes
checking for an ANSI C-conforming const... yes
checking for inline... inline
checking whether time.h and sys/time.h may both be included... yes
checking for size_t... yes
checking for suitable m4... m4
checking how to switch to text section... .text
checking how to export a symbol... .globl
checking what assembly label suffix to use... :
checking if globals are prefixed by underscore... no
checking how to switch to text section... (cached) .text
checking how to export a symbol... (cached) .globl
checking for assembler .type directive... 
checking for working alloca.h... yes
checking for alloca... yes
checking for ANSI C header files... (cached) yes
checking math.h usability... yes
checking math.h presence... yes
checking for math.h... yes
checking limits.h usability... yes
checking limits.h presence... yes
checking for limits.h... yes
checking malloc.h usability... yes
checking malloc.h presence... yes
checking for malloc.h... yes
checking for strings.h... (cached) yes
checking sys/time.h usability... yes
checking sys/time.h presence... yes
checking for sys/time.h... yes
checking for unistd.h... (cached) yes
checking io.h usability... no
checking io.h presence... no
checking for io.h... no
checking signal.h usability... yes
checking signal.h presence... yes
checking for signal.h... yes
checking fcntl.h usability... yes
checking fcntl.h presence... yes
checking for fcntl.h... yes
checking windows.h usability... no
checking windows.h presence... no
checking for windows.h... no
checking ctype.h usability... yes
checking ctype.h presence... yes
checking for ctype.h... yes
checking for sys/types.h... (cached) yes
checking sys/resource.h usability... yes
checking sys/resource.h presence... yes
checking for sys/resource.h... yes
checking for working strtod... yes
checking for pow in -lm... yes
checking for floor in -lm... yes
checking for sqrt in -lm... yes
checking for fmod in -lm... yes
checking for cos in -lm... yes
checking for cblas_dgemm in -lgslcblas... no
checking for isascii... yes
checking for memset... yes
checking for strchr... yes
checking for strlen... yes
checking for strncasecmp... yes
checking for strstr... yes
checking for access... yes
checking for unlink... yes
checking for isspace... yes
checking for isdigit... yes
checking for isxdigit... yes
checking for time... yes
checking for ctime... yes
checking for setpriority... yes
checking for nice... yes
checking for gethostname... yes
checking for gettimeofday... yes
checking for getrusage... yes
checking for memmove... yes
checking for signal... yes
checking for fcntl... yes
checking for fileno... yes
checking for malloc_usable_size... yes
checking gmp.h usability... yes
checking gmp.h presence... yes
checking for gmp.h... yes
checking for recent GMP... no
configure: error: GMP 5.0.0 or newer is required

[EdH]

I know you have a lot of time into the procedure you're following, but if you'd like to take a look at a different set of manual steps for linux, I have a step-by-step procedure at:

Steps to install and set up Aliqueit on an Ubuntu* computer

edit: Just a note that yafu is one of the packages on the way to Aliqueit. You don't have to go that far.

If you would like to try these, I will be watching this thread to help you and, if you find trouble spots, you can help me clarify/correct my instructions.

Even if you don't wish to follow my steps, you might find a bit of info on the page to help with your current work.

[Dubslow]

Shouldn't the ./configure also have a --with-gmp=<dir> option added to it? Seems like a bug in the script again to me?

[2147483647]

Originally Posted by EdH

I know you have a lot of time into the procedure you're following, but if you'd like to take a look at a different set of manual steps for linux, I have a step-by-step procedure at:

Steps to install and set up Aliqueit on an Ubuntu* computer

Is this possible to use without any "make install" commands? I'm not an admin on this machine so I can't run any of them.

Originally Posted by Dubslow

Shouldn't the ./configure also have a --with-gmp=<dir> option added to it? Seems like a bug in the script again to me?

Just tried this but only get gmp-6.0.0/include does not exist and then it immediately stops.

[EdH]

Originally Posted by 2147483647

Is this possible to use without any "make install" commands? I'm not an admin on this machine so I can't run any of them
...

I believe you can use absolute paths if you haven't installed gmp and ecm, but for now I would say, don't try. I don't know how to tell msieve where to find the libraries, maybe in the Makefile. If you don't have m4 and gcc installed you would need privileges to get them.

If I get a chance today, I'll see if I can find a machine to try it without installations and note the changes. Maybe you can get the other procedure working by then. Is the lack of admin privileges what's hindering you currently?

I haven't looked lately, but I thought all the packages were available already compiled, especially for Windows. Dubslow would know more about that, though.

As a totally different approach, can you set up a Virtual Machine under your account in which you can install Ubuntu? I have done that with full success and I don't think I needed admin privileges on the main account. But, a Virtual Machine Manager would have to be installed. You may not have that either.

[2147483647]

Originally Posted by EdH

If I get a chance today, I'll see if I can find a machine to try it without installations and note the changes.

Thanks, that would be great.

gcc is already installed and I'm pretty sure m4 is too so that shouldn't be a problem.

For the other procedure, there seem to be a lot of bugs/errors/strange things in the setup so I'll probably just abandon it.

Yes there are pre-compiled windows binaries that I've been using for a while, but I'm interested in playing around with some of the code to try and distribute ggnfs over lots of computers automatically.

[EdH]

Originally Posted by 2147483647

...
Yes there are pre-compiled windows binaries that I've been using for a while, but I'm interested in playing around with some of the code to try and distribute ggnfs over lots of computers automatically.

I'm actually doing that, although I awoke today to find that my factmsieve.py is refusing to run for some strange reason and I don't have time to figure out why right now due to snow...

Basically, I have two machines running the assignments for all the others. The first one is a Raspberry Pi that assigns ECM tasks and the second is a core2 quad that assigns the gnfs jobs. It's very inefficient ATM and (obviously) prone to failure.

The two "assignment" hosts write a file to the other to signal when to swap back and forth. The clients are running two different scripts; one for the ECM tasks and one for the gnfs tasks. They are totally free to exist or not without affecting the hosts. I do have duplication at about 20% for the gnfs work and once the job is handed off to the gnfs machine and it finds a poly, ECM is stopped on all the clients, so I do miss a bit of the higher end ECMing.