Error compiling version 29.1 on Arch Linux

[graysky]

I can't seem to compile version 29.1 on Arch Linux. The make step is failing with:

/usr/bin/ld: cannot find -lhwloc
collect2: error: ld returned 1 exit status
make: *** [makefile:43: mprime] Error 1

It seems like I'm missing 'libhwloc.pc' which isn't provided by any upstream package. If I run the following, it tells me as much:

% pkg-config --static --libs libhwloc
Package libhwloc was not found in the pkg-config search path.
Perhaps you should add the directory containing `libhwloc.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libhwloc' found

I modified linux64/makefile as follows (perhaps a tweak here can be the solution):

CFLAGS = -I.. -I../gwnum -DX86_64 -O2 -Wno-unused-result
LFLAGS = -Wl,-M
LIBS = ../gwnum/gwnum.a ../gwnum/gwnum.ld -lm -lpthread -Wl,-Bstatic $(shell pkg-config --static --libs hwloc) -Wl,-Bstatic $(shell pkg-config --static --libs libcurl) -lstdc++ -Wl,-Bdynamic -ldl

Any suggestions are welcomed.

Complete make output:

% make clean && make -j16
rm -f mprime  prime.o menu.o
[ ! -e ../security.h ] && touch ../security.h || true
gcc -I.. -I../gwnum -DX86_64 -O2 -Wno-unused-result -c prime.c
gcc -I.. -I../gwnum -DX86_64 -O2 -Wno-unused-result -c menu.c
[ ! -e ../security.c ] && touch ../security.c || true
[ ! -e ../secure5.c ] && touch ../secure5.c || true
gcc -Wl,-M -o mprime prime.o menu.o factor64.o ../gwnum/gwnum.a ../gwnum/gwnum.ld -lm -lpthread -Wl,-Bstatic -lhwloc -lm -lnuma -lltdl -lpthread -ldl -Wl,-Bstatic -lcurl -lssh2 -lpsl -lssl -lcrypto -lssl -lcrypto -lgssapi_krb5 -lkrb5 -lk5crypto -lcom_err -lz -lstdc++ -Wl,-Bdynamic -ldl
Archive member included to satisfy reference by file (symbol)

../gwnum/gwnum.a(gianthlp.o)  prime.o (muladdhlp)
../gwnum/gwnum.a(cpuidhlp.o)  prime.o (erdtsc)
../gwnum/gwnum.a(cpuid.o)     prime.o (one_hundred_thousand_clocks)
../gwnum/gwnum.a(gwnum.o)     prime.o (gwinit2)
../gwnum/gwnum.a(gwtables.o)  ../gwnum/gwnum.a(gwnum.o) (pow_two_above_or_equal)
../gwnum/gwnum.a(gwthread.o)  prime.o (gwmutex_init)
../gwnum/gwnum.a(gwini.o)     prime.o (IniFileReread)
../gwnum/gwnum.a(gwutil.o)    ../gwnum/gwnum.a(gwnum.o) (aligned_offset_malloc)
../gwnum/gwnum.a(gwdbldbl.o)  ../gwnum/gwnum.a(gwnum.o) (gwasm_constants)
../gwnum/gwnum.a(giants.o)    prime.o (allocgiant)
../gwnum/gwnum.a(ymult3ax.o)  ../gwnum/gwnum.a(gwnum.o) (gwyaddq3)
../gwnum/gwnum.a(ymult3a_fma3.o)
                              ../gwnum/gwnum.a(gwnum.o) (yr3FMA3)
../gwnum/gwnum.a(ymult3a_core.o)
                              ../gwnum/gwnum.a(gwnum.o) (yr3CORE)
../gwnum/gwnum.a(ymult3.o)    ../gwnum/gwnum.a(gwnum.o) (ygw_carries_wpn3)
../gwnum/gwnum.a(ymult1ax_fma3.o)
                              ../gwnum/gwnum.a(gwnum.o) (yr1FMA3)
../gwnum/gwnum.a(ymult1ax_core.o)
                              ../gwnum/gwnum.a(gwnum.o) (gwyaddq1)
../gwnum/gwnum.a(xmult3ax.o)  ../gwnum/gwnum.a(gwnum.o) (gwxadd3)
../gwnum/gwnum.a(xmult3a_k8.o)
                              ../gwnum/gwnum.a(gwnum.o) (xr3K8)
../gwnum/gwnum.a(xmult3a_core.o)
                              ../gwnum/gwnum.a(gwnum.o) (xr3CORE)
../gwnum/gwnum.a(xmult2ax.o)  ../gwnum/gwnum.a(gwnum.o) (gwxaddq2)
../gwnum/gwnum.a(xmult2a_k8.o)
                              ../gwnum/gwnum.a(gwnum.o) (xr2K8)
../gwnum/gwnum.a(xmult2a_core.o)
                              ../gwnum/gwnum.a(gwnum.o) (xr2CORE)
../gwnum/gwnum.a(xmult2.o)    ../gwnum/gwnum.a(gwnum.o) (pass1_aux_entry_point)
../gwnum/gwnum.a(xmult1ax.o)  ../gwnum/gwnum.a(gwnum.o) (gwxaddq1)
../gwnum/gwnum.a(mult.o)      ../gwnum/gwnum.a(gwnum.o) (gwinfo1)
../gwnum/gwnum.a(hg_p4.o)     ../gwnum/gwnum.a(mult.o) (xfft_hg_10K_ip_8_4_P4)
../gwnum/gwnum.a(hg_k8.o)     ../gwnum/gwnum.a(mult.o) (xfft_hg_10K_ip_8_4_K8)
../gwnum/gwnum.a(hg_k10.o)    ../gwnum/gwnum.a(mult.o) (xfft_hg_10K_ip_8_4_K10)
../gwnum/gwnum.a(hg_core.o)   ../gwnum/gwnum.a(mult.o) (xfft_hg_10K_ip_8_4_CORE)
../gwnum/gwnum.a(hg_blend.o)  ../gwnum/gwnum.a(mult.o) (xfft_hg_32_op_BLEND)
../gwnum/gwnum.a(yr4dwpn_fma3.o)
                              ../gwnum/gwnum.a(mult.o) (yfft_r4dwpn_6K_48_2_FMA3)
../gwnum/gwnum.a(yr4dwpn_core.o)
                              ../gwnum/gwnum.a(mult.o) (yfft_r4dwpn_6K_48_2_CORE)
../gwnum/gwnum.a(yr4dwpn5_fma3.o)
                              ../gwnum/gwnum.a(mult.o) (yfft_r4dwpn_1440K_4608_2_FMA3)
../gwnum/gwnum.a(yr4dwpn5_core.o)
                              ../gwnum/gwnum.a(mult.o) (yfft_r4dwpn_3200K_12800_1_CORE)
../gwnum/gwnum.a(yr4dwpn3_fma3.o)
                              ../gwnum/gwnum.a(mult.o) (yfft_r4dwpn_1M_12_2_FMA3)
../gwnum/gwnum.a(yr4dwpn3_core.o)
                              ../gwnum/gwnum.a(mult.o) (yfft_r4dwpn_1M_12_4_CORE)
../gwnum/gwnum.a(yr4_fma3.o)  ../gwnum/gwnum.a(mult.o) (yfft_r4_32_op_FMA3)
../gwnum/gwnum.a(yr4_core.o)  ../gwnum/gwnum.a(mult.o) (yfft_r4_32_op_CORE)
../gwnum/gwnum.a(r4dwpn_p4.o)
                              ../gwnum/gwnum.a(mult.o) (xfft_r4dwpn_32K_8_4_P4)
../gwnum/gwnum.a(r4dwpn_k8.o)
                              ../gwnum/gwnum.a(mult.o) (xfft_r4dwpn_32K_8_2_K8)
../gwnum/gwnum.a(r4dwpn_k10.o)
                              ../gwnum/gwnum.a(mult.o) (xfft_r4dwpn_64K_8_2_K10)
../gwnum/gwnum.a(r4dwpn_core.o)
                              ../gwnum/gwnum.a(mult.o) (xfft_r4dwpn_32K_8_2_CORE)
../gwnum/gwnum.a(r4dwpn5_p4.o)
                              ../gwnum/gwnum.a(mult.o) (xfft_r4dwpn_1152K_4608_4_P4)
../gwnum/gwnum.a(r4dwpn5_k8.o)
                              ../gwnum/gwnum.a(mult.o) (xfft_r4dwpn_5376K_12288_4_K8)
../gwnum/gwnum.a(r4dwpn5_k10.o)
                              ../gwnum/gwnum.a(mult.o) (xfft_r4dwpn_1728K_4608_4_K10)
../gwnum/gwnum.a(r4dwpn5_core.o)
                              ../gwnum/gwnum.a(mult.o) (xfft_r4dwpn_1152K_4608_4_CORE)
../gwnum/gwnum.a(r4dwpn3_p4.o)
                              ../gwnum/gwnum.a(mult.o) (xfft_r4dwpn_1M_12_4_P4)
../gwnum/gwnum.a(r4dwpn3_k8.o)
                              ../gwnum/gwnum.a(mult.o) (xfft_r4dwpn_576K_2304_4_K8)
../gwnum/gwnum.a(r4dwpn3_k10.o)
                              ../gwnum/gwnum.a(mult.o) (xfft_r4dwpn_1M_12_4_K10)
../gwnum/gwnum.a(r4dwpn3_core.o)
                              ../gwnum/gwnum.a(mult.o) (xfft_r4dwpn_576K_2304_4_CORE)
../gwnum/gwnum.a(r4delay_p4.o)
                              ../gwnum/gwnum.a(mult.o) (xfft_r4delay_64K_8_4_P4)
../gwnum/gwnum.a(r4delay_k8.o)
                              ../gwnum/gwnum.a(mult.o) (xfft_r4delay_32K_8_2_K8)
../gwnum/gwnum.a(r4delay_k10.o)
                              ../gwnum/gwnum.a(mult.o) (xfft_r4delay_64K_8_2_K10)
../gwnum/gwnum.a(r4delay_core.o)
                              ../gwnum/gwnum.a(mult.o) (xfft_r4delay_32K_8_2_CORE)
../gwnum/gwnum.a(r4delay5_p4.o)
                              ../gwnum/gwnum.a(mult.o) (xfft_r4delay_1152K_4608_4_P4)
../gwnum/gwnum.a(r4delay5_k8.o)
                              ../gwnum/gwnum.a(mult.o) (xfft_r4delay_3456K_9216_1_K8)
../gwnum/gwnum.a(r4delay5_k10.o)
                              ../gwnum/gwnum.a(mult.o) (xfft_r4delay_1728K_4608_4_K10)
../gwnum/gwnum.a(r4delay5_core.o)
                              ../gwnum/gwnum.a(mult.o) (xfft_r4delay_1152K_4608_4_CORE)
../gwnum/gwnum.a(r4delay3_p4.o)
                              ../gwnum/gwnum.a(mult.o) (xfft_r4delay_576K_2304_4_P4)
../gwnum/gwnum.a(r4delay3_k8.o)
                              ../gwnum/gwnum.a(mult.o) (xfft_r4delay_576K_2304_4_K8)
../gwnum/gwnum.a(r4delay3_k10.o)
                              ../gwnum/gwnum.a(mult.o) (xfft_r4delay_1M_12_4_K10)
../gwnum/gwnum.a(r4delay3_core.o)
                              ../gwnum/gwnum.a(mult.o) (xfft_r4delay_576K_2304_4_CORE)
../gwnum/gwnum.a(r4_p4.o)     ../gwnum/gwnum.a(mult.o) (xfft_r4_16K_8_4_P4)
../gwnum/gwnum.a(r4_k8.o)     ../gwnum/gwnum.a(mult.o) (xfft_r4_56K_8_1_K8)
../gwnum/gwnum.a(r4_k10.o)    ../gwnum/gwnum.a(mult.o) (xfft_r4_28K_8_2_K10)
../gwnum/gwnum.a(r4_core.o)   ../gwnum/gwnum.a(mult.o) (xfft_r4_20K_8_4_CORE)
../gwnum/gwnum.a(xmult3.o)    ../gwnum/gwnum.a(r4dwpn_p4.o) (xgw_carries_wpn)
/usr/bin/ld: cannot find -lhwloc
collect2: error: ld returned 1 exit status
make: *** [makefile:43: mprime] Error 1

[graysky]

Ah, I fixed it with these modifications to linux64/makefile:

CFLAGS = -I.. -I../gwnum -march=x86-64 -DX86_64 -O2 -Wno-unused-result
CPPFLAGS = -I.. -I../gwnum -march=x86-64 -O2 -Wno-unused-result
LIBS = ../gwnum/gwnum.a ../gwnum/gwnum.ld -lm -lpthread $(shell pkg-config --libs libcurl) -lstdc++ $(shell pkg-config --static --libs hwloc)

AUR package: https://aur.archlinux.org/packages/mprime/

[Dubslow]

George, isn't hwloc supposed to be static by default? The error indicates it attempted to link dynamically.

[Prime95]

I link hwloc using -Bstatic

[graysky]

Originally Posted by Prime95

I link hwloc using -Bstatic

I'm not sure I fully understand the implications of setting up the changes to the makefile to allow a build on Arch Linux. For me, it builds with or without the -Bstatic swtich in LIBS. Would you please review the following changes that I have made to enable building on Arch Linux for sanity?

--- a/linux64/makefile	2017-03-18 21:18:04.000000000 -0400
+++ b/linux64/makefile	2017-03-20 09:38:23.273509623 -0400
@@ -21,13 +21,13 @@
 #	LIBS = ../gwnum/gwnum.a ../gwnum/gwnum.ld -lm -lpthread -Wl,-Bstatic $(shell pkg-config --static --libs libhwloc) -Wl,-Bstatic $(shell pkg-config --static --libs libcurl) -lstdc++ -Wl,-Bdynamic -ldl
 
 CC = gcc
-CFLAGS = -I.. -I../gwnum -I/usr/local/include -DX86_64 -O2 
+CFLAGS = -I.. -I../gwnum -march=x86-64 -DX86_64 -O2 -Wno-unused-result
 
 CPP = g++
-CPPFLAGS = -I.. -I../gwnum -DX86_64 -O2
+CPPFLAGS = -I.. -I../gwnum -march=x86-64 -O2 -Wno-unused-result
 
 LFLAGS = -Wl,-M -Wl,-L/usr/local/lib
-LIBS = ../gwnum/gwnum.a ../gwnum/gwnum.ld -lm -lpthread -Wl,-Bstatic -lhwloc -Wl,-Bstatic -lcurl -Wl,-Bdynamic -lrt -lstdc++ -lz -lxml2 -ldl
+LIBS = ../gwnum/gwnum.a ../gwnum/gwnum.ld -lm -lpthread $(shell pkg-config --libs libcurl) -lstdc++ $(shell pkg-config --static --libs hwloc) -Bstatic
 
 FACTOROBJ = factor64.o
 LINUXOBJS = prime.o menu.o

For reference:

% pkg-config --libs libcurl
-lcurl

% pkg-config --static --libs hwloc
-lhwloc -lm -lnuma -lltdl -lpthread -ldl

[Explorer09]

May I ask a question here? Since libhwloc is intended to link statically in mprime, and that hwloc has many optional components that may be enabled or disabled at configure time...

What are the minimum features needed to enable in hwloc in order to let mprime work?

Let me state this: I did read the makefile in mprime 29.1 source, and it suggest to "--enable-static --disable-shared --disable-pci --disable-cuda --disable-gl", but my question is, what about the other features? Are they needed to be enabled, optional, or not needed at all?

These are the hwloc features that I'm currently confused about:

--enable-cpuid (x86 cpuid feature)
--enable-libxml2 (this requires libxml2 and a pkg-config entry for it)
--enable-libnuma (this depends on libnuma)
--enable-libudev (this depends on libudev)
--enable-opencl (this depends on libOpenCL)
--enable-nvml (this depends on libnvidia-ml)

For now, I only enable the cpuid feature and explicitly disable the rest because I don't like additional library dependency because I am trying to cross-compile mprime. It would be better to explicitly mention which are needed and which aren't in mprime's makefile.