mersenneforum.org Call for help
 Register FAQ Search Today's Posts Mark Forums Read

 2005-07-07, 18:04 #1 Wacky     Jun 2003 The Texas Hill Country 32·112 Posts Call for help Although we are spread extremely thin, some progress is being made in getting new sievers operational. However, I could certainly use some help. First, in the code area, I have two fairly small tasks that someone could perform without either a significant knowledge of NFS or a large or ongoing commitment of time. a) A regression testing program, written in C or C++ that can verify that the output file from a test run "matches" a reference copy. (I can presently do this on Unix-style systems using awk/sed/diff or perl. However, I need something readily portable to many test systems) b) Bob Silverman has some MP Math routines for x86 machines. These need to be reformatted to compile under gcc. This is a translation task, not a programming task, per se. The second area where I could use some help is in the area of test machines. As we are able to get the various sievers running, we need to be able to compile and test for a wide range of system setups. Although we are not yet to that point, I would like to be able to test a wide range of CPU/OS combinations. The setup is fairly well automated. You maintain a copy to the sources on your hard disk. A scheduled task runs a program each night (or other time of the day) to update the sources from our CVS repository and recompile the changed routines. It then runs some tests and reports the results back to my server which posts a summary. By examining the errors and/or timing reports, we can track the effect of code changes on machine configurations that we otherwise would not have access. To participate, you would need a C compiler development environment and the willingness to participate on a continuing basis. If you would like to help in either area, please e-mail me. Last fiddled with by Wacky on 2005-07-07 at 18:18
2005-07-08, 17:21   #2
Jeff Gilchrist

Jun 2003

3·17·23 Posts

Quote:
 Originally Posted by Wacky a) A regression testing program, written in C or C++ that can verify that the output file from a test run "matches" a reference copy. (I can presently do this on Unix-style systems using awk/sed/diff or perl. However, I need something readily portable to many test systems)
How closely do they have to match? Identical output files or more refined than that?

If identical, why not just take the MD5/SHA-1 hash of the files to see if they match?

Jeff.

 2005-07-08, 17:39 #3 Wacky     Jun 2003 The Texas Hill Country 32×112 Posts How close. Identical EXCEPT .... We need to be able to supply the two files to compare and a definition of the acceptably mismatched lines. For example, a typical ASCII Factor Base has a line or two that contain information about where and when the file was generated. I'm happy to simply skip the matching of those 2 or 3 lines. If it is necessary to go to a full programming language, I'd probably suggest Perl. But I prefer to avoid that dependency.
2005-07-08, 19:07   #4
xilman
Bamboozled!

"𒉺𒌌𒇷𒆷𒀭"
May 2003
Down not across

22·11·257 Posts

Quote:
 Originally Posted by Jeff Gilchrist How closely do they have to match? Identical output files or more refined than that? If identical, why not just take the MD5/SHA-1 hash of the files to see if they match? Jeff.
I observe that Perl5 is freely available for almost all systems, and is much less system-dependent than awk. Cygwin awk, for instance, rarely gave the same results as BSD awk when I tried to port earlier NFSNET code.

sed and its friends seem not to be as widespread as Perl.

Paul

2005-07-08, 19:30   #5
Wacky

Jun 2003
The Texas Hill Country

32·112 Posts

Quote:
 Originally Posted by xilman I observe that Perl5 is freely available for almost all systems, and is much less system-dependent than awk. Cygwin awk, for instance, rarely gave the same results as BSD awk when I tried to port earlier NFSNET code. sed and its friends seem not to be as widespread as Perl. Paul
Paul,
I do not disagree with your statement. That is why I said that if it got to a point where we really needed that level of generality, I would probably choose Perl over other alternatives.

However, there are two problems with Perl:

It is TOO general. That has implications upon data integrity and security. (For example, you might write to the wrong file) A less general tool that does limited things based on its input is less likely to do the wrong thing because of some input data error.

Although it is "freely available", that does not mean that it IS present. Requiring it is just one more setup hurdle that I think we can avoid.

Richard

Last fiddled with by Wacky on 2005-07-08 at 19:33

 2005-07-08, 19:31 #6 akruppa     "Nancy" Aug 2002 Alexandria 246710 Posts What rights on part of the user does the test suite need? I.e. is root access required to install it (for cron jobs or such)? The only PC I personally own is a P3-500 running Linux which is probably neither a targeted, nor a rare platform. At the University, the only machine I can access freely are x86 (Athlon XP mostly) Linux boxes, a few Opteron 150 running Linux, and some UltraSparc 3 and 4 under SunOS 5.9. The Opteron and Sparc are machines for general use by students so I better get permission before I start scheduled jobs or they won't stay for long. If any of these would be any use to you, let me know. Alex
2005-07-08, 19:46   #7
Wacky

Jun 2003
The Texas Hill Country

32×112 Posts

Quote:
 Originally Posted by akruppa What rights on part of the user does the test suite need? I.e. is root access required to install it (for cron jobs or such)? The only PC I personally own is a P3-500 running Linux which is probably neither a targeted, nor a rare platform. At the University, the only machine I can access freely are x86 (Athlon XP mostly) Linux boxes, a few Opteron 150 running Linux, and some UltraSparc 3 and 4 under SunOS 5.9. The Opteron and Sparc are machines for general use by students so I better get permission before I start scheduled jobs or they won't stay for long. If any of these would be any use to you, let me know. Alex
It would need no special user permissions, assuming that the general user can compile and execute an arbitrary program. Some systems do require higher access to set up the running of scheduled jobs. Others permit a normal user to run scheduled jobs under their own id. YMMV.

As for the UltraSparc, access to them for the purpose might be of interest if there are more than one or two out there that would want to sieve on a regular basis. However, Dan reports that they have phased out all of theirs and I don't know of anyone in particular that has a number of them which would participate if we had a binary available. However, I will keep you in mind should the need arise.

Richard

Last fiddled with by Wacky on 2005-07-08 at 20:19

 2005-07-11, 09:25 #8 Mystwalker     Jul 2004 Potsdam, Germany 83110 Posts I could write an application for a) using Java. Portability shouldn't be an issue, but (at least outside of the Windows world), I guess it's not more often pre-installed than Perl. In addition, when it is the only Java application and it needs to be run many times, loading issues might impact performance. OTOH, if it is used frequently, maybe a good option would be to let it stay resident and hand over work instead or starting it each time...
2005-07-11, 13:13   #9
R.D. Silverman

Nov 2003

746010 Posts

Quote:
 Originally Posted by Wacky Although we are spread extremely thin, some progress is being made in getting new sievers operational. However, I could certainly use some help. First, in the code area, I have two fairly small tasks that someone could perform without either a significant knowledge of NFS or a large or ongoing commitment of time. a) A regression testing program, written in C or C++ that can verify that the output file from a test run "matches" a reference copy. (I can presently do this on Unix-style systems using awk/sed/diff or perl. However, I need something readily portable to many test systems) b) Bob Silverman has some MP Math routines for x86 machines. These need to be reformatted to compile under gcc. This is a translation task, not a programming task, per se. The second area where I could use some help is in the area of test machines. As we are able to get the various sievers running, we need to be able to compile and test for a wide range of system setups. Although we are not yet to that point, I would like to be able to test a wide range of CPU/OS combinations. The setup is fairly well automated. You maintain a copy to the sources on your hard disk. A scheduled task runs a program each night (or other time of the day) to update the sources from our CVS repository and recompile the changed routines. It then runs some tests and reports the results back to my server which posts a summary. By examining the errors and/or timing reports, we can track the effect of code changes on machine configurations that we otherwise would not have access. To participate, you would need a C compiler development environment and the willingness to participate on a continuing basis. If you would like to help in either area, please e-mail me.

Once upon a time doing this kind of work was my job.
Now I have a real-world job.

I will be happy to provide guidance/advice/discussion to anyone working
on a port of my lattice siever and underlying X86 64-bit arithmetic code, but
I no longer have time to do the port myself. C'est la vie.

 2005-07-11, 13:30 #10 cjohnsto   Jun 2005 3·5 Posts How about the following ANSI C code for comparisons. Sorry about the lack of comments, it should be fairly obvious what I have done. I have tested it a bit but programs can always use more. Especialy those written at midnight as quickly as possible. usage is program file1 file2 [skipfile] where skipfile contains a sorted list of line numbers (on seperate lines) to skip in both file1 and file2. If this isn't what you wanted tell me why and what you do want. Code: #include #include int main(int argc, char* argv[]) { FILE *skipfile = NULL, *in1 = NULL, *in2 = NULL; int skip = -1; char line1[1024], line2[1024]; int lineno = 0; int skipped = 1; if ((argc < 3) || (argc > 4)) { fprintf(stderr, "usage: %s file1 file2 [skipfile]\nskipfile contains line numbers on seperate lines that will not be compared between file1 and file2 this file must be sorted\n", argv[0]); return 1; } if (argc == 4) { skipfile = fopen(argv[3], "r"); if (!skipfile) { fprintf(stderr, "Could not open %s\n", argv[3]); return 2; } } in1 = fopen(argv[1], "r"); in2 = fopen(argv[2], "r"); if (!in1) { fprintf(stderr, "could not open %s\n", argv[1]); return 3; } if (!in2) { fprintf(stderr, "could not open %s\n", argv[2]); return 4; } while(1) { int eol = 0; lineno++; if (skipfile && skipped) { if (fgets(line1, 1024, skipfile) != NULL) { skip = atoi(line1); skipped = 0; } } while (!eol) { int i; char* res1 = fgets(line1, 1024, in1); char* res2 = fgets(line2, 1024, in2); if (!res1 || !res2) { if (feof(in1) || feof(in2)) { if (feof(in1) && feof(in2)) { fprintf(stderr, "Files are Identical\n"); return 0; } else { fprintf(stderr, "Files differ in length\n"); if (feof(in1)) printf("%s is shorter\n", argv[1]); if (feof(in2)) printf("%s is shorter\n", argv[2]); return -1; } } else { fprintf(stderr, "Error occured reading one of the files\n"); return 5; } } for (i = 0; ; ++i) { if ((line1[i] == 0) || (line2[i] == 0)) { if (skip == lineno) { break; } else if ((line1[i] == 0) && (line2[i] == 0)) break; else { fprintf(stderr, "Difference on line %d\n", lineno); printf("%s\n%s\n", line1, line2); return -1; } } else if ((line1[i] == '\n') || (line2[i] == '\n')) { if (skip == lineno) { skipped = 1; eol = 1; break; } else if ((line1[i] == '\n') && (line2[i] == '\n')) { eol = 1; break; } else { fprintf(stderr, "Difference on line %d\n", lineno); printf("%s\n%s\n", line1, line2); return -1; } } else if (skip == lineno) { continue; } else if (line1[i] == line2[i]) { continue; } else { fprintf(stderr, "Difference on line %d\n", lineno); printf("%s\n%s\n", line1, line2); return -1; } } } } }
2005-07-12, 03:35   #11
Wacky

Jun 2003
The Texas Hill Country

32×112 Posts

Quote:
 Originally Posted by R.D. Silverman I will be happy to provide guidance/advice/discussion to anyone working on a port of my lattice siever and underlying X86 64-bit arithmetic code, but I no longer have time to do the port myself. C'est la vie.
Bob,
Thanks for the offer. I did not ever consider asking you to do this because, IMHO, if you have any time available for NFS sieving, your time can be better spent on improving the code rather than a task as mundane as translating a page of assembler code from one "dialect" to another. You have already provided the appropriate lines of code. The only problem is that I need to also be able to compile those lines on a different compiler. Hopefully we can get this done soon.

 Similar Threads Thread Thread Starter Forum Replies Last Post ThomRuley Factoring 14 2019-05-25 00:12 ET_ Teams 6 2018-12-18 12:08 schickel Science & Technology 6 2010-04-18 10:36 davieddy Puzzles 19 2009-10-06 05:41 em99010pepe No Prime Left Behind 2 2008-11-26 13:21

All times are UTC. The time now is 10:39.

Mon May 23 10:39:07 UTC 2022 up 39 days, 8:40, 0 users, load averages: 1.36, 1.13, 1.08