Compiling YAFU under MinGW

wombatman · 2016-01-14, 14:23

I'm trying to build the latest and greatest YAFU under MinGW-64. I can actually get it to build with and without NFS=1 flag (by compiling MSieve without GPU poly search). However, when I go to run tune() as a test on it, I get a crash immediately following the relation-gathering step for the C60 number.

To confirm that it's not the relation-gathering step, I can delete siqs.dat and watch the relations get built again. So it's the step immediately after. I've tried the following build steps (with make clean before each):

Code:

make x86_64 USE_SSE41=1 NFS=1
make x86_64 NFS=1
make x86_64 USE_SSE41=1
make x86_64

The crash is some variation on:

Code:

Unhandled exception at 0x0000000076F7FFC2 (ntdll.dll) in yafu-x64.exe: 0xC0000374: A heap has been corrupted (parameters: 0x0000000076FF7470).

Any ideas?

bsquared · 2016-01-14, 15:07

Can you post the screen output when running with -v, so I can see where it crashes exactly?

wombatman · 2016-01-14, 16:24

I can when I get home, absolutely. I'm going to try and build it here on my laptop at work since I have a little free time today. If I get the same error, I'll post the screenshot of that.

wombatman · 2016-01-15, 01:31

It's not a screenshot, but here's the full text:

Code:

$ yafu-x64.exe "tune()" -v -v


01/14/16 19:30:18 v1.34.5 @ BEN-PC, System/Build Info:
Using GMP-ECM, Powered by GMP
detected        Intel(R) Core(TM) i7-4930K CPU @ 3.40GHz
detected L1 = 32768 bytes, L2 = 12582912 bytes, CL = 64 bytes
measured cpu frequency ~= 3362.986750
using 1 random witnesses for Rabin-Miller PRP checks

===============================================================
======= Welcome to YAFU (Yet Another Factoring Utility) =======
=======             bbuhrow@gmail.com                   =======
=======     Type help at any time, or quit to quit      =======
===============================================================
cached 78498 primes. pmax = 999983


>> Setting THREADS = 1 for tuning

starting SIQS on c60: 349594255864176572614071853194924838158088864370890996447417
fb bounds: 85%
        small: 1024
        SPV: 36
        10bit: 104
        11bit: 168
        12bit: 296
        13bit: 520
        32k div 3: 688
        14bit: 968
        15bit: 1736
        med: 2544
        large: 3488
        all: 3488
start primes
        SPV: 251
        10bit: 1039
        11bit: 2153
        12bit: 4201
        13bit: 8147
        32k div 3: 11003
        14bit: 16433
        15bit: 32803
        med: 49277
        large: 70009

==== sieve params ====
n = 61 digits, 201 bits
factor base: 3488 primes (max prime = 70009)
single large prime cutoff: 3850495 (55 * pmax)
allocating 2 large prime slices of factor base
buckets hold 2048 elements
using SSE4.1 enabled 32k sieve core
sieve interval: 4 blocks of size 32768
polynomial A has ~ 7 factors
using multiplier of 7
using SPV correction of 20 bits, starting at offset 36
using SSE2 for x64 sieve scanning
using SSE2 for resieving 13-16 bit primes
using SSE2 for 8x trial divison to 13 bits
using SSE4.1 and inline ASM for small prime sieving
using SSE2 for poly updating up to 15 bits
using SSE4.1 for medium prime poly updating
using SSE4.1 and inline ASM for large prime poly updating
trial factoring cutoff at 68 bits

==== sieving in progress (1 thread):    3552 relations needed ====
====           Press ctrl-c to abort and save state           ====
996 rels found: 700 full + 296 from 6611 partial, (5387.61 rels/sec)
Max specified relations found

sieve time = 0.0000, relation time = 0.0000, poly_time = 0.0000
trial division touched 114375 sieve locations out of 1593573376


sieving required 6079 total polynomials
trial division touched 114375 sieve locations out of 1593573376

That last line is the last thing that appears on the screen before it crashes.

Edit to add: If I run siqs() on the number, the crash occurs after the linear algebra step:

Code:

sieving required 0 total polynomials
trial division touched 0 sieve locations out of 0
QS elapsed time = 0.0400 seconds.

==== post processing stage (msieve-1.38) ====
begin with 18896 relations
reduce to 5518 relations in 2 passes
attempting to read 5518 relations
recovered 5518 relations
recovered 4324 polynomials
freed 2 duplicate relations
attempting to build 3804 cycles
found 3804 cycles in 1 passes
distribution of cycle lengths:
   length 1 : 1830
   length 2 : 1974
largest cycle: 2 relations
matrix is 3488 x 3804 (0.5 MB) with weight 98453 (25.88/col)
sparse part has weight 98453 (25.88/col)
filtering completed in 4 passes
matrix is 3232 x 3296 (0.4 MB) with weight 82450 (25.02/col)
sparse part has weight 82450 (25.02/col)
commencing Lanczos iteration
memory use: 0.6 MB
lanczos halted after 52 iterations (dim = 3226)
recovered 61 nontrivial dependencies
Lanczos elapsed time = 0.0700 seconds.
Sqrt elapsed time = 0.0050 seconds.
SIQS elapsed time = 0.1150 seconds.

wombatman · 2016-01-15, 15:39

To add something that might help, the binary from sourceforge goes through tune() without issue on the same computer.

I also took a whack at compiling YAFU via VS2012, but the project kept incorporating the AVX2 code, even without the appropriate flag. Can you (or someone else) advise on how I might stop that from happening?

bsquared · 2016-01-15, 17:16

The latest SVN is likely a mess for compiling on anything other than linux. That is my bad. I abuse SVN by checking in code that doesn't necessarily work everywhere, because I thought I was the only person that used the head revision in SVN. MSVC builds in particular have been broken for a while now.

wombatman · 2016-01-15, 17:45

No problem. It builds without error under MinGW--I can do, for instance, ecm factoring and the like without any issue. I haven't tried an nfs(), but I'll do that and report back. Maybe I can at least narrow down what's going on for you.

Edit: nfs() on a small (91 digits) number works fine all the way through. And to confirm that it's siqs() related, I ran the same number through there--it crashed after building and processing the matrix:

Code:

largest cycle: 17 relations
matrix is 74160 x 74841 (18.5 MB) with weight 4256736 (56.88/col)
sparse part has weight 4256736 (56.88/col)
filtering completed in 3 passes
matrix is 67465 x 67529 (16.8 MB) with weight 3861614 (57.18/col)
sparse part has weight 3861614 (57.18/col)
saving the first 48 matrix rows for later
matrix is 67417 x 67529 (14.5 MB) with weight 3347595 (49.57/col)
sparse part has weight 3112970 (46.10/col)
matrix includes 64 packed rows
using block size 27011 for processor cache size 12288 kB
commencing Lanczos iteration
memory use: 11.6 MB
lanczos halted after 1067 iterations (dim = 67415)
recovered 17 nontrivial dependencies
Lanczos elapsed time = 12.1930 seconds.
Sqrt elapsed time = 0.0680 seconds.
SIQS elapsed time = 303.6504 seconds.

Last edit for now: The crash occurs at free_siqs(thread_data[0].sconf); in siqs.c (as confirmed by strategically placed printf() lines). So it looks like something with the pthread library. I'll futz around more with it--good task for a sick day.

wombatman · 2016-01-16, 20:43

Ok! Figured out where the crash is along with a crude bandage that seems to work for now.

The offending line is Line 2782 of SIQS.c in the free_siqs() function:

Code:

free(sconf->modsqrt_array);

Again with my slightly above-average printf statement usage, this line was isolated. If I comment it out, everything works perfectly and factors are returned. The factors were confirmed as correct using FactorDB.

I'll leave it to those of you who are far smarter than I am as to what actually causes the error and what a proper fix that is not simply sweeping it under the rug would be.

bsquared · 2016-01-16, 23:23

Nice work finding a workaround, albeit non-ideal. I will see if I can find out what's going on.

ET_ · 2016-01-21, 18:17

Quote:

Originally Posted by wombatman

Ok! Figured out where the crash is along with a crude bandage that seems to work for now.

The offending line is Line 2782 of SIQS.c in the free_siqs() function:

Code:

free(sconf->modsqrt_array);

Again with my slightly above-average printf statement usage, this line was isolated. If I comment it out, everything works perfectly and factors are returned. The factors were confirmed as correct using FactorDB.

I'll leave it to those of you who are far smarter than I am as to what actually causes the error and what a proper fix that is not simply sweeping it under the rug would be.

Were you using the new -std=c11 flag of gcc? The last revision of gcc seems to use a new thread library, not compatible with pthreads...

wombatman · 2016-01-21, 19:48

I'm not, no. Or, at least, not on purpose. I don't recall seeing any std flag during compilation.

2016-01-14, 14:23	#1
wombatman I moo ablest echo power! May 2013 73C₁₆ Posts	Compiling YAFU under MinGW I'm trying to build the latest and greatest YAFU under MinGW-64. I can actually get it to build with and without NFS=1 flag (by compiling MSieve without GPU poly search). However, when I go to run tune() as a test on it, I get a crash immediately following the relation-gathering step for the C60 number. To confirm that it's not the relation-gathering step, I can delete siqs.dat and watch the relations get built again. So it's the step immediately after. I've tried the following build steps (with make clean before each): Code: make x86_64 USE_SSE41=1 NFS=1 make x86_64 NFS=1 make x86_64 USE_SSE41=1 make x86_64 The crash is some variation on: Code: Unhandled exception at 0x0000000076F7FFC2 (ntdll.dll) in yafu-x64.exe: 0xC0000374: A heap has been corrupted (parameters: 0x0000000076FF7470). Any ideas?

2016-01-15, 17:45	#7
wombatman I moo ablest echo power! May 2013 1852₁₀ Posts	No problem. It builds without error under MinGW--I can do, for instance, ecm factoring and the like without any issue. I haven't tried an nfs(), but I'll do that and report back. Maybe I can at least narrow down what's going on for you. Edit: nfs() on a small (91 digits) number works fine all the way through. And to confirm that it's siqs() related, I ran the same number through there--it crashed after building and processing the matrix: Code: largest cycle: 17 relations matrix is 74160 x 74841 (18.5 MB) with weight 4256736 (56.88/col) sparse part has weight 4256736 (56.88/col) filtering completed in 3 passes matrix is 67465 x 67529 (16.8 MB) with weight 3861614 (57.18/col) sparse part has weight 3861614 (57.18/col) saving the first 48 matrix rows for later matrix is 67417 x 67529 (14.5 MB) with weight 3347595 (49.57/col) sparse part has weight 3112970 (46.10/col) matrix includes 64 packed rows using block size 27011 for processor cache size 12288 kB commencing Lanczos iteration memory use: 11.6 MB lanczos halted after 1067 iterations (dim = 67415) recovered 17 nontrivial dependencies Lanczos elapsed time = 12.1930 seconds. Sqrt elapsed time = 0.0680 seconds. SIQS elapsed time = 303.6504 seconds. Last edit for now: The crash occurs at free_siqs(thread_data[0].sconf); in siqs.c (as confirmed by strategically placed printf() lines). So it looks like something with the pthread library. I'll futz around more with it--good task for a sick day. Last fiddled with by wombatman on 2016-01-15 at 18:44

2016-01-16, 20:43	#8
wombatman I moo ablest echo power! May 2013 2²×463 Posts	Ok! Figured out where the crash is along with a crude bandage that seems to work for now. The offending line is Line 2782 of SIQS.c in the free_siqs() function: Code: free(sconf->modsqrt_array); Again with my slightly above-average printf statement usage, this line was isolated. If I comment it out, everything works perfectly and factors are returned. The factors were confirmed as correct using FactorDB. I'll leave it to those of you who are far smarter than I am as to what actually causes the error and what a proper fix that is not simply sweeping it under the rug would be.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Compiling YAFU for Windows using mingw	bsquared	YAFU	87	2022-11-21 22:31
Running YAFU via Aliqueit doesn't find yafu.ini	EdH	YAFU	8	2018-03-14 17:22
Guide to compiling YAFU	Mr. Odd	YAFU	4	2017-04-24 15:40
Need help compiling YAFU from SVN.	Stargate38	YAFU	14	2016-01-20 21:46
Better with MinGW 5.0.3?	M0CZY	GMP-ECM	1	2006-07-16 07:29

2016-01-14, 15:07	#2
bsquared "Ben" Feb 2007 3732₁₀ Posts	Can you post the screen output when running with -v, so I can see where it crashes exactly?

2016-01-14, 16:24	#3
wombatman I moo ablest echo power! May 2013 2²·463 Posts	I can when I get home, absolutely. I'm going to try and build it here on my laptop at work since I have a little free time today. If I get the same error, I'll post the screenshot of that.

2016-01-15, 15:39	#5
wombatman I moo ablest echo power! May 2013 1852₁₀ Posts	To add something that might help, the binary from sourceforge goes through tune() without issue on the same computer. I also took a whack at compiling YAFU via VS2012, but the project kept incorporating the AVX2 code, even without the appropriate flag. Can you (or someone else) advise on how I might stop that from happening?

2016-01-15, 17:16	#6
bsquared "Ben" Feb 2007 2²×3×311 Posts	The latest SVN is likely a mess for compiling on anything other than linux. That is my bad. I abuse SVN by checking in code that doesn't necessarily work everywhere, because I thought I was the only person that used the head revision in SVN. MSVC builds in particular have been broken for a while now.

2016-01-16, 23:23	#9
bsquared "Ben" Feb 2007 2²·3·311 Posts	Nice work finding a workaround, albeit non-ideal. I will see if I can find out what's going on.

2016-01-21, 19:48	#11
wombatman I moo ablest echo power! May 2013 73C₁₆ Posts	I'm not, no. Or, at least, not on purpose. I don't recall seeing any std flag during compilation.