mtsieve

[chalsall]

Originally Posted by storm5510

I awakened very early this morning to find the latest srsieve2cl has caused a reboot. Not sure why. It may be in the system logs. I haven't looked yet.

Please rest.

The kit can wait.

[storm5510]

Originally Posted by chalsall

Please rest.

The kit can wait.

Kit?

[chalsall]

Originally Posted by storm5510

Kit?

LOL... Glad to see you feeling better.

Kit is slang. It means the equipment you regularly work with. Or, in a military context, what you carry.

Nowadays it means the compute you spin up beside you. Or, near you. Sometimes "in the cloud".

Just to share... I hate it when people say "in the cloud", because usually they have no idea what the cloud actually is.

To share... People who think AI is going to end the world might want to study a bit about how neural networks work.

[storm5510]

Originally Posted by chalsall

LOL... Glad to see you feeling better...

Just to share... I hate it when people say "in the cloud", because usually they have no idea what the cloud actually is.

Off-topic: Other than a mild headache and sneezing, it is mostly gone. A less then 24-hour deal. I get these now and then. A cloud system can be in a chicken house behind a barn in a rural area. It does not float. A server-farm more likely. It is a totally misapplied description.

[rogue]

Originally Posted by ryanp

Another bug in srsieve2?

$ ./srsieve2 -P 4e14 -o out.txt -W 32 -s "11*2^n+1" -n 15e6 -N 25e6
srsieve2 v1.6.6, a program to find factors of k*b^n+c numbers for fixed b and variable k and n
Sieving with generic logic for p >= 3
Sieve started: 3 < p < 4e14 with 10000001 terms (15000000 < n < 25000000, k*2^n+1) (expecting 9673251 factors)
Sieving with single sequence c=1 logic for p >= 257
BASE_MULTIPLE = 30, POWER_RESIDUE_LCM = 720, LIMIT_BASE = 720
Fatal Error:  Expected 15 subsequences but 233 were created

Interesting. This is the first time I have seen this get triggered. I wonder if this is something I accidentally introduced in the latest build for single sequences.

[Trying2Sieve]

Noob here. Kind of. I haven't found a large prime in about 10 years and I see so much has changed. Last one I found was ranked 84 on the list (2013-ish).

I now have access to a ridiculous number of cores and would like to start my esoteric prime hunting again.

I have a great custom parallelizer tool for NewPGen formatted output, but NewPGen is very slow.

For me, the mtsieve output file is confusing. I can't determine the exponents being tested by cursory inspection.

Is there a way to link the speed of mtsieve with the NewpGen output?

I tried the -N setting to no avail.

Any reply that helps will be greatly appreciated.

[rogue]

Originally Posted by ryanp

Another bug in srsieve2?

$ ./srsieve2 -P 4e14 -o out.txt -W 32 -s "11*2^n+1" -n 15e6 -N 25e6
srsieve2 v1.6.6, a program to find factors of k*b^n+c numbers for fixed b and variable k and n
Sieving with generic logic for p >= 3
Sieve started: 3 < p < 4e14 with 10000001 terms (15000000 < n < 25000000, k*2^n+1) (expecting 9673251 factors)
Sieving with single sequence c=1 logic for p >= 257
BASE_MULTIPLE = 30, POWER_RESIDUE_LCM = 720, LIMIT_BASE = 720
Fatal Error:  Expected 15 subsequences but 233 were created

I have fixed this and committed code changes. I ran into a segfault with multiple sequences when using Legendre tables. This does not appear to be new. I would like to fix it before posting a new build.

[rogue]

Originally Posted by Trying2Sieve

Noob here. Kind of. I haven't found a large prime in about 10 years and I see so much has changed. Last one I found was ranked 84 on the list (2013-ish).

I now have access to a ridiculous number of cores and would like to start my esoteric prime hunting again.

I have a great custom parallelizer tool for NewPGen formatted output, but NewPGen is very slow.

For me, the mtsieve output file is confusing. I can't determine the exponents being tested by cursory inspection.

Is there a way to link the speed of mtsieve with the NewpGen output?

I tried the -N setting to no avail.

Did you try to use -h to see what options are available for the program you are using?

Go here. The default output for many sieves is ABCD format because it is compact. Some sieves do not support the ABCD format. The ABCD format is supported by pfgw. I do not know if llr supports that format off the top of my head. Some sieves have a -f option that allows you to specify the format of the output. As newpgen is not supported and since the most popular sieves it supports are either in the mtsieve framework or somewhere else, it is not recommended to use that format. The ABC format, which llr and pfgw both support, is preferred as the header of those files is not as cryptic as newpgen. The pfgw readme explains the ABC and ABCD formats. ABC is really easy to understand compared to newpgen.

[kruoli]

When I try to continue a file I presieved with srsieve2 to 1e10 with srsieve2cl and Legendre tables, I get:
Fatal Error: Expected 81 subsequences but 967 were created
Command line:
srsieve2cl -p 1e10 -i remaining.abcd -g 16 -Q -o remaining_new.abcd -H -l {any value greater than 0}

Additionally, it takes really long to start the sieve. I ommitted -l. It used more than 20 GB RAM and has not started sieving after 20 minutes. I was trying to sieve the 77 remaining sequences of R53. Maybe I was doing something wrong?

The initial sieve was called as:
srsieve2 -n 100e3 -N 250e3 -P 1e10 -W 16 -Q -o remaining.abcd -s k.in
k.in held all k's in a line each.

[rogue]

Originally Posted by kruoli

When I try to continue a file I presieved with srsieve2 to 1e10 with srsieve2cl and Legendre tables, I get:
Fatal Error: Expected 81 subsequences but 967 were created
Command line:
srsieve2cl -p 1e10 -i remaining.abcd -g 16 -Q -o remaining_new.abcd -H -l {any value greater than 0}

Additionally, it takes really long to start the sieve. I ommitted -l. It used more than 20 GB RAM and has not started sieving after 20 minutes. I was trying to sieve the 77 remaining sequences of R53. Maybe I was doing something wrong?

The initial sieve was called as:
srsieve2 -n 100e3 -N 250e3 -P 1e10 -W 16 -Q -o remaining.abcd -s k.in
k.in held all k's in a line each.

The bug I have to fix is with multiple sequences and using Legendre tables.

I will take a look to see why it is using so much memory and task so long to load. It shouldn't take that long.

[Trying2Sieve]

Originally Posted by rogue

Did you try to use -h to see what options are available for the program you are using?

Go here. The default output for many sieves is ABCD format because it is compact. Some sieves do not support the ABCD format. The ABCD format is supported by pfgw. I do not know if llr supports that format off the top of my head. Some sieves have a -f option that allows you to specify the format of the output. As newpgen is not supported and since the most popular sieves it supports are either in the mtsieve framework or somewhere else, it is not recommended to use that format. The ABC format, which llr and pfgw both support, is preferred as the header of those files is not as cryptic as newpgen. The pfgw readme explains the ABC and ABCD formats. ABC is really easy to understand compared to newpgen.

Ok, I understand, there is a large base of support for the ABCD format.
Maybe I can do the equivalent in that format, (if I understood it) so let me show you what I am doing and maybe it will give you guys an idea or two also.

7 40
7 45
7 48
7 68
7 80
7 83
7 97
7 119
7 124
7 129
7 130

That's the sample NewPGen code to find primes that start with 7, end in 1, and have 00000...00000 in the middle.
The number next to the 7 is the power of 10 that will be used, like 10^45, 10^48 etc.

It's a vertical text file, and each line will be read one at a time, as you all know and expect.

My "parallelizer" basically turns this into a huge rectangular grid of data points.

Something like this:

7 40	7 45	7 48	7 68	7 80	7 83	7 97
7 273	7 278	7 282	7 293	7 311	7 336	7 352
7 545	7 560	7 563	7 564	7 566	7 568	7 578
7 780	7 808	7 810	7 823	7 824	7 827	7 832
7 1082	7 1083	7 1091	7 1092	7 1106	7 1118	7 1123
7 1314	7 1317	7 1320	7 1346	7 1350	7 1352	7 1379
7 1526	7 1542	7 1546	7 1559	7 1565	7 1566	7 1569
7 1680	7 1682	7 1690	7 1701	7 1705	7 1721	7 1722
7 1940	7 1942	7 1943	7 1958	7 1964	7 1978	7 1980
7 2154	7 2156	7 2160

Each VERTICAL column of candidate primes will be given to a different core.

In this small example, I used 30-way parallelism to demonstrate its usefulness.

Candidate prime #2 on core one is actually #31 in the master sieve list.
Candidate prime #3 on core one is actually #61 in the master sieve list.

It's like "skipping ahead" 30 candidates at a time, or making the artificial sieve depth for that core nearly 97% better. I'll encounter larger primes more quickly on average, even if they are scattered haphazardly, because I am not testing them "in order."

With the ABCD format, I have no idea how to do this.

Or is this just a waste of time?

I'm sure I'll encounter larger primes quicker this way (I have 365 cores actually) so each day of searching should statistically be equivalent to one year's worth of probing on a single core.

How can I do this in ABCD format?

And thanks Rogue, your contributions here have been enormous and undoubtedly everyone vastly appreciates your effort.