mersenneforum.org  

Go Back   mersenneforum.org > Factoring Projects > YAFU

Reply
 
Thread Tools
Old 2012-07-16, 15:56   #45
jcrombie
 
jcrombie's Avatar
 
"Jonathan"
Jul 2010
In a tangled web...

5×43 Posts
Default

First off, thanks Ben for an amazing program (and open source and public domain)!

I tried the pre-compiled yafu 1.32 and it works just fine. When I compiled it on my system ( Ubuntu 12.04 LTS) I get this runtime crash immediately on start up.

It happens on Line 847 in driver.c with a "buffer overflow detected" error message.

Code:
        sysname_sz = 255; 
        ret = gethostname(sysname,sysname_sz);
This is apparently caused by the sysname string being 80 bytes long. (Line 169 yafu.h)

Code:
    char sysname[80];
and passing 255 as the string size in gethostname().

The fix should be a simple changing of the 80 to 256.
I also suggest null terminating the string afterwards since gethostname() doesn't actually guarantee to do it.

Code:
                sysname[255] = '\0'; 
                sysname_sz = strlen(sysname);
Cheers,

Jonathan
jcrombie is offline   Reply With Quote
Old 2012-07-16, 16:18   #46
bsquared
 
bsquared's Avatar
 
"Ben"
Feb 2007

22×3×293 Posts
Default

Good catch, thanks!

Since your hostname is apparently long enough to test this, can you let me know if your proposed fix doesn't work?

I wonder why the pre-compiled version doesn't crash...
bsquared is offline   Reply With Quote
Old 2012-07-16, 17:42   #47
jcrombie
 
jcrombie's Avatar
 
"Jonathan"
Jul 2010
In a tangled web...

110101112 Posts
Default

Quote:
Originally Posted by bsquared View Post
Good catch, thanks!

Since your hostname is apparently long enough to test this, can you let me know if your proposed fix doesn't work?

I wonder why the pre-compiled version doesn't crash...
Actually I have a really short hostname "Octacore".

Code:
                sysname[255] = '\0'; 
        sysname_sz = strlen(sysname);
printf("sysname == (%s) and sysname_sz == (%d)\n", sysname, sysname_sz );
Code:
jcrombie@Octacore:~/yafu/yafu-1.32$ ./yafu
sysname == (Octacore) and sysname_sz == (8)


07/16/12 11:32:47 v1.32 @ Octacore, System/Build Info: 
Using GMP-ECM 5.0.1, Powered by GMP 5.0.1
detected AMD FX(tm)-8150 Eight-Core Processor           
detected L1 = 16384 bytes, L2 = 8388608 bytes, CL = 64 bytes
measured cpu frequency ~= 3612.038900
using 20 random witnesses for Rabin-Miller PRP checks
....
I've tested the fix on this computer and it works.

Let me know if you need any other info.
jcrombie is offline   Reply With Quote
Old 2012-07-16, 19:06   #48
chalsall
If I May
 
chalsall's Avatar
 
"Chris Halsall"
Sep 2002
Barbados

976710 Posts
Default

Quote:
Originally Posted by jcrombie View Post
The fix should be a simple changing of the 80 to 256.
I also suggest null terminating the string afterwards since gethostname() doesn't actually guarantee to do it.
Just being really pedantic here, but a better fix would be:

Code:
ret = gethostname(sysname,sizeof(sysname) / sizeof(*sysname));
sysname[sizeof(sysname) / sizeof(*sysname)] = 0;
Otherwise you're relying on the programmer to remember to change the length of the array in two (or more) locations....

P.S. Sorry... really bored at the moment. LIME (my ISP) have lost a large portion of the Internet thanks to screwed up BGP.
chalsall is offline   Reply With Quote
Old 2012-07-16, 19:31   #49
jcrombie
 
jcrombie's Avatar
 
"Jonathan"
Jul 2010
In a tangled web...

110101112 Posts
Default

Quote:
Originally Posted by chalsall View Post
Just being really pedantic here, but a better fix would be:

Code:
ret = gethostname(sysname,sizeof(sysname) / sizeof(*sysname));
sysname[sizeof(sysname) / sizeof(*sysname)] = 0;
Otherwise you're relying on the programmer to remember to change the length of the array in two (or more) locations....

P.S. Sorry... really bored at the moment. LIME (my ISP) have lost a large portion of the Internet thanks to screwed up BGP.
Yes, very flexible and elegant. Unfortunately, I think spot a new bug.
sizeof(sysname) == 80
sizeof(*sysname) == 1
80 / 1 == 80
sysname[80] would access the 81st byte which would be off the end of the array.

sysname[sizeof(sysname)/sizeof(*sysname)-1] should work.
[ Edit: This is better: sysname[(sizeof(sysname)-1)/sizeof(*sysname)] ]

BGP?

Cheers

Last fiddled with by jcrombie on 2012-07-16 at 19:46 Reason: Superior solution
jcrombie is offline   Reply With Quote
Old 2012-07-16, 19:46   #50
chalsall
If I May
 
chalsall's Avatar
 
"Chris Halsall"
Sep 2002
Barbados

100110001001112 Posts
Default

Quote:
Originally Posted by jcrombie View Post
sysname[sizeof(sysname)/sizeof(*sysname)-1] should work.
Yeah, you're correct. My code was wrong.

Quote:
Originally Posted by jcrombie View Post
BGP?
Border Gateway Protocol. It's how big ISPs and carriers exchange routing information.

There's a reason LIME (really Cable and Wireless) is known as Clueless and Worthless here in the Caribbean....
chalsall is offline   Reply With Quote
Old 2012-07-17, 04:37   #51
jcrombie
 
jcrombie's Avatar
 
"Jonathan"
Jul 2010
In a tangled web...

5×43 Posts
Default

Hi again.

I've located a few places where there is an fopen() without a corresponding fclose().

factor_common.c Line 2002 & Line 2042
SIQS.c Line 128

Last week I was using the batchfile option and I would always get a crash around the 550th item. Sometimes the error message was "too many open files". Checking today, I see that there were hundreds of "siqs.dat" files open while yafu was executing. Adding the fclose()'s fixed the problem.

Cheers
jcrombie is offline   Reply With Quote
Old 2012-07-17, 12:44   #52
bsquared
 
bsquared's Avatar
 
"Ben"
Feb 2007

DBC16 Posts
Default

Thank you! Much appreciated.
bsquared is offline   Reply With Quote
Old 2012-07-21, 03:35   #53
swellman
 
swellman's Avatar
 
Jun 2012

60218 Posts
Default

I recently saw some strange behavior - ran a C150 on an old notebook (i3 processor, 32 bit Windoze, etc). Built up about 10M relations. Then I hit control-c, let YAFU do its shutdown thing and return to c: prompt. I copied NSF.dat, NSF.job, and the last_specQ file to a much faster machine (64 bit Windoze).

Restarted nfs() using the -R flag on this faster pc. YAFU could not determine the last spec_Q value (there was some gibberish displayed in the cmd window once YAFU had digested the nfs.dat file), and YAFU started using the default value, i.e. it started duplicating the 10M relations I had already found. So I closed the cmd window, deleted any recently created files and tried nfs() again, this time using the -ns flag and specifying the correct spec_Q value over a small range of spec_Q (1000). This seemed to work. Once this finished, I submitted the nfs() -R again. This time YAFU found the correct last spec_Q value but only recognized 5M relations. I let this run and expect it to complete ~July 26.

So

1. Is there a tendency for 32 bit and 64 bit created data files to not play well together?

2. The control-c command seems to exit YAFU cleanly, but I've experienced gibberish at the end of nfs.dat a few times upon restart.

3. Why is the Last_specQ file seemingly ignored? YAFU seems intent on searching the NSF.dat file for the largest spec_Q value.

4. Once seiving is complete, will my 44M+ relations with the "missing" 5M relations yield a good matrix and (eventually) factors? I admit I'm kind of curious to see what happens.

I have the data files available - I can attempt to recreate the behavior and post output later if necessary.

In the past I've used the technique of running nfs() on my old machine then moving files to a fast machine to complete nfs() and it went flawlessly. Maybe it was user error or just a random problem (it happens).

Any insights?

FWIW, because my old box is so slow, I may relegate it to poly searches only. It's seiving output is so low it's really not worth using.
swellman is online now   Reply With Quote
Old 2012-07-21, 04:06   #54
Dubslow
Basketry That Evening!
 
Dubslow's Avatar
 
"Bunslow the Bold"
Jun 2011
40<A<43 -89<O<-88

3×29×83 Posts
Default

I've never heard of a last_specQ file on Linux, that's why it searches the last lines of nfs.dat.

When you hit ^C, GGNFS should terminate, but YAFU still puts all those rels into NFS.dat.

It sounds like somehow the dump of the terminated sieve job rels got corrupted somehow. The full 10M rels should have all been in the one NFS.dat file.

FWIW, I've never run into corruption on interruption (on Linux though like I said).
Dubslow is offline   Reply With Quote
Old 2012-07-21, 04:20   #55
bsquared
 
bsquared's Avatar
 
"Ben"
Feb 2007

66748 Posts
Default

There's another reason I don't use the .last_spq file (other than portability): it gets clobbered during multi-threaded jobs. That is, each ggnfs instance outputs to the same file so you can't trust any of its contents.

The other points:
1) I haven't heard of a problem before, and I can't think of why there would be an issue.
2) Is the "gibberish" truly random crap printed to the screen, or is there some structure to it? I ask because, when resuming, yafu will print the last several lines that it found in the .dat file along with some of its internal conversation about deciding whether the lines refer to algebraic side or rational side special-q. I've never had cause to use this output yet, but haven't got around to disabling it.

tl;dr: The gibberish is probably normal.
4) Yes, I would expect so. However, no warranty is expressed or implied, nausea, vomiting, or hallucinations are possible side effects, void where prohibited, do not eat, etc.
bsquared is offline   Reply With Quote
Reply

Thread Tools


Similar Threads
Thread Thread Starter Forum Replies Last Post
Running YAFU via Aliqueit doesn't find yafu.ini EdH YAFU 8 2018-03-14 17:22
Where to report bugs Matt Software 1 2007-02-20 19:13
Possible Prime95 bugs JuanTutors Software 9 2006-09-24 21:22
RMA 1.7 beta bugs TTn 15k Search 2 2004-11-24 22:11
RMA 1.6 fixes LLR bugs! TTn 15k Search 16 2004-06-16 01:22

All times are UTC. The time now is 00:07.


Mon Aug 2 00:07:50 UTC 2021 up 9 days, 18:36, 0 users, load averages: 1.19, 1.36, 1.34

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.