Using msieve with c

[burrobert]

I recently set up gmp, gmp-ecm, ggnfs msieve, and aliquiet on my Ubuntu 12.04 machine. Everything seems to work fine from the command line and the python script factsieve.py does what it should. I decided to try using the msieve functions in a c program but get a segmentation fault. The program works fine on my Macbook under xcode. Here is the program (with a test factorisation of 101101).

#include <stdio.h>
#include <stdlib.h>
#include <math.h>
#include <gmp.h>
#include <msieve.h>
#include <sys/times.h>
 
int main (int argc, const char * argv[])
{
msieve_obj *mobj;
char *mint;
char buffer[1000]; 
 
srand((unsigned int) time( NULL ));
mint = "101101";
printf("mint = %s\n", mint);
msieve_factor *f;
mobj = calloc(1, sizeof(msieve_obj));
 
mobj->savefile.name = "/home/msieve_savefile.txt";
mobj->savefile.buf = &buffer[0];
mobj->max_relations = 0;
mobj->input = mint;
mobj->seed1 = rand();
mobj->seed2 = rand();
 
msieve_run(mobj);
f = mobj->factors;
// Makes sure there is a place to start
if (f != 0 ) {
while ( f->next != 0 ) {
printf("%s, %s\n", f->number, (f->factor_type == 0) ? "composite" : (f->factor_type == 1) ? "prime" : "probable prime");
f = f->next;
}
printf("%s, %s\n", f->number, (f->factor_type == 0) ? "composite" : (f->factor_type == 1) ? "prime" : "probable prime");
}
 
return EXIT_SUCCESS;
}

*******************
I get the output:

mint = 101101
Segmentation fault (core dumped)

I have tracked the problem down to the function msieve_run. Valgrind output is as follows:

mint = 101101
==2987== Invalid write of size 1
==2987== at 0x8050EDD: mp_print (in /home/rob/Mathwork/msieve/trunk/a.out)
==2987== by 0x804CCC5: msieve_run (in /home/rob/Mathwork/msieve/trunk/a.out)
==2987== by 0x804C061: main (in /home/rob/Mathwork/msieve/trunk/a.out)
==2987== Address 0x400 is not stack'd, malloc'd or (recently) free'd
==2987== 
==2987== 
==2987== Process terminating with default action of signal 11 (SIGSEGV)
==2987== Access not within mapped region at address 0x400
==2987== at 0x8050EDD: mp_print (in /home/rob/Mathwork/msieve/trunk/a.out)
==2987== by 0x804CCC5: msieve_run (in /home/rob/Mathwork/msieve/trunk/a.out)
==2987== by 0x804C061: main (in /home/rob/Mathwork/msieve/trunk/a.out)

I am not sure what might be causing the problems so would be grateful for any suggestions.

[EdH]

If you set up things using my Steps to install and set up Aliqueit on an Ubuntu computer, did you note the mention of using -march=native in the Makefile? If not, you might give that a try. The note is the last item under the "d. Msieve" step.I'd be interested if this works...

[Dubslow]

It looks like the multiple-precision code written for Msieve (and now mostly replaced by GMP) is failing.

You should note that a) that code was written to work for an "average" NFS number, which is many more than a few digits; b) Msieve refuses to run NFS of any sort on a number small than 60-80 digits or so, even if the scratch-MP code didn't crash.

My hunch is that since the MP code was "ad hoc", it's not anywhere near as bug-free as GMP, meaning it probably assumes an input of at least 32 bits or some such. (jasonp can comment on this, since it's his code )

Try using a number above the NFS limit. (Even an 85 digit job should be less than an hour on most computers.)

PS Use the [code] tag for your code.

PPS There might be some element of your msieve_obj that's uninitialized, since you didn't use msieve_obj_new().

msieve_obj * msieve_obj_new(char *input_integer, uint32 flags,
	                            char *savefile_name, char *logfile_name,
	                            char *nfs_fbfile_name,
	                            uint32 seed1, uint32 seed2, uint32 max_relations,
	                            enum cpu_type cpu,
	                            uint32 cache_size1, uint32 cache_size2,
	                            uint32 num_threads, uint32 which_gpu, 
	                            const char *nfs_args) {
	
	        msieve_obj *obj = (msieve_obj *)xcalloc((size_t)1, sizeof(msieve_obj));
	
	        obj->input = input_integer;
	        obj->flags = flags;
	        obj->seed1 = seed1;
	        obj->seed2 = seed2;
	        obj->max_relations = max_relations;
	        obj->cpu = cpu;
	        obj->cache_size1 = cache_size1;
	        obj->cache_size2 = cache_size2;
	        obj->num_threads = num_threads;
	        obj->which_gpu = which_gpu;
	        obj->logfile_name = MSIEVE_DEFAULT_LOGFILE;
	        obj->nfs_args = nfs_args;
	        if (logfile_name)
	                obj->logfile_name = logfile_name;
	        obj->nfs_fbfile_name = MSIEVE_DEFAULT_NFS_FBFILE;
	        if (nfs_fbfile_name)
	                obj->nfs_fbfile_name = nfs_fbfile_name;
	        obj->mp_sprintf_buf = (char *)xmalloc(32 * MAX_MP_WORDS + 1);
	        savefile_init(&obj->savefile, savefile_name);
	        
	        return obj;
	}

In particular, some important things that need to be initialized are `mp_sprintf_buf`, `num_threads`, `logfile_name`, and `nfs_fbfile_name`. I'd recommend you just call this function instead of doing the initialization yourself.

[burrobert]

Originally Posted by EdH

If you set up things using my Steps to install and set up Aliqueit on an Ubuntu computer, did you note the mention of using -march=native in the Makefile? If not, you might give that a try. The note is the last item under the "d. Msieve" step.I'd be interested if this works...

I recompiled msieve with the -march=native flag but received the same error. Thanks for providing the setup instructions. I found then very useful and easy to follow.

[burrobert]

Thanks for your suggestions Dubslow. I firstly tried running the code on a larger integer but this produced the same error. Your second suggestion however was correct. I used the msieve_obj_new() function to create and initialise the msieve_obj and this removed the problem. It is interesting though that there was no problem on my Macbook Pro both when I compiled and ran the program in xcode and when I did it from the command line. I guess as a general principle though it is better to use a provided initialiser. Thanks for your help.

[EdH]

Originally Posted by burrobert

I recompiled msieve with the -march=native flag but received the same error. Thanks for providing the setup instructions. I found then very useful and easy to follow.

Thanks for checking that for me. And, for the feedback on the Steps...

[Dubslow]

Originally Posted by burrobert

It is interesting though that there was no problem on my Macbook Pro both when I compiled and ran the program in xcode and when I did it from the command line. I guess as a general principle though it is better to use a provided initialiser.

It could be that a different compiler (clang?) automatically initializes all C variables to 0/NULL, as most other languages do, while gcc doesn't. (When you say command line, do you mean with gcc as opposed to clang?)

[Batalov]

Originally Posted by burrobert

Thanks for your suggestions Dubslow. I firstly tried running the code on a larger integer but this produced the same error. Your second suggestion however was correct. I used the msieve_obj_new() function to create and initialise the msieve_obj and this removed the problem. It is interesting though that there was no problem on my Macbook Pro both when I compiled and ran the program in xcode and when I did it from the command line. I guess as a general principle though it is better to use a provided initialiser. Thanks for your help.

The demo.c code is deliberately outside of the library, so that all the API was clear; it can be read and cut-and-pasted. You cannot just call the core methods (from any library, not just libmsieve) without the proper initialization.

Also, generally it is not a robust idea to manually set up some variables when the constructor (in this case, msieve_obj_new()) exists. Over the course of time, the library can be later updated, methods refactored, etc, and the new constructor will take care of that, but the raw code will fail. That is all of course unless the role of the raw code is for one day/one test only and then to be deleted.

[burrobert]

Originally Posted by Dubslow

It could be that a different compiler (clang?) automatically initializes all C variables to 0/NULL, as most other languages do, while gcc doesn't. (When you say command line, do you mean with gcc as opposed to clang?)

Yes that's right. It worked with both gcc and clang on my Macbook.

[burrobert]

Originally Posted by Batalov

The demo.c code is deliberately outside of the library, so that all the API was clear; it can be read and cut-and-pasted. You cannot just call the core methods (from any library, not just libmsieve) without the proper initialization.

Also, generally it is not a robust idea to manually set up some variables when the constructor (in this case, msieve_obj_new()) exists. Over the course of time, the library can be later updated, methods refactored, etc, and the new constructor will take care of that, but the raw code will fail. That is all of course unless the role of the raw code is for one day/one test only and then to be deleted.

Actually I hadn't noticed the demo.c program but I'll have a look at it now. I wasn't able to find any documentation for the API so I looked at the header files to work out the structures and functions.