mersenneforum.org  

Go Back   mersenneforum.org > Extra Stuff > Programming

Reply
 
Thread Tools
Old 2022-05-10, 11:11   #1
pepi37
 
pepi37's Avatar
 
Dec 2011
After milion nines:)

3×7×73 Posts
Default Script for split file

I need script that will "ask" me number of workers and then split input file equally into worktodo.txt file ( and if possible add "Worker line") ( in this case 3 workers)


Quote:
example

input.txt
PRP=1,2,3
PRP=1,2,4
PRP=1,2,5


output

[Worker #1]
PRP=1,2,3
[Worker #2]
PRP=1,2,4
[Worker #3]
PRP=1,2,5

Thanks
pepi37 is online now   Reply With Quote
Old 2022-05-10, 11:17   #2
paulunderwood
 
paulunderwood's Avatar
 
Sep 2002
Database er0rr

2·5·421 Posts
Default

Quote:
Originally Posted by pepi37 View Post
I need script that will "ask" me number of workers and then split input file equally into worktodo.txt file ( and if possible add "Worker line") ( in this case 3 workers)
Windows or Linux? What is your preferred (scripting) language?
paulunderwood is online now   Reply With Quote
Old 2022-05-10, 13:38   #3
kruoli
 
kruoli's Avatar
 
"Oliver"
Sep 2017
Porta Westfalica, DE

24·5·13 Posts
Default

In bash:
Code:
if [ $# -ne 3 ];
then
  echo 'Usage: bash splitter.sh [in file] [out file] [number of workers]';
  exit -1;
fi;
if ls "$1."* 1> /dev/null 2>&1;
then
  echo 'Temporary files could not be created.';
  exit -2;
fi;
range=$(seq 1 $3);
if [ $? -ne 0 ];
then
  echo 'The worker count was not numerical.';
  exit -3;
fi;

for w in $range;
do
  echo "[Worker #$w]" >> "$1.tmp";
done;
cat "$1" >> "$1.tmp";
split -n r/$3 "$1.tmp" "$1.s";
cat "$1.s"* > "$2";
rm "$1."*;
kruoli is online now   Reply With Quote
Old 2022-05-10, 13:45   #4
pepi37
 
pepi37's Avatar
 
Dec 2011
After milion nines:)

3·7·73 Posts
Default

Quote:
Originally Posted by paulunderwood View Post
Windows or Linux? What is your preferred (scripting) language?
Whatever I can run under Linux ๐Ÿ˜€
Language, everyone you like it. I will adopt
Thanks
pepi37 is online now   Reply With Quote
Old 2022-05-10, 13:54   #5
Uncwilly
6809 > 6502
 
Uncwilly's Avatar
 
"""""""""""""""""""
Aug 2003
101ร—103 Posts

1061610 Posts
Default

What about this tool? https://www.mersenne.ca/balance.php
Uncwilly is offline   Reply With Quote
Old 2022-05-10, 14:37   #6
pepi37
 
pepi37's Avatar
 
Dec 2011
After milion nines:)

3×7×73 Posts
Default

Quote:
Originally Posted by Uncwilly View Post
At first sight look perfect!
pepi37 is online now   Reply With Quote
Old 2022-05-10, 22:07   #7
pepi37
 
pepi37's Avatar
 
Dec 2011
After milion nines:)

27758 Posts
Default

Quote:
Originally Posted by Uncwilly View Post
But doesnot work on PRP tests...
pepi37 is online now   Reply With Quote
Old 2022-05-10, 22:22   #8
pepi37
 
pepi37's Avatar
 
Dec 2011
After milion nines:)

3×7×73 Posts
Default

Quote:
Originally Posted by kruoli View Post
In bash:
Code:
if [ $# -ne 3 ];
then
  echo 'Usage: bash splitter.sh [in file] [out file] [number of workers]';
  exit -1;
fi;
if ls "$1."* 1> /dev/null 2>&1;
then
  echo 'Temporary files could not be created.';
  exit -2;
fi;
range=$(seq 1 $3);
if [ $? -ne 0 ];
then
  echo 'The worker count was not numerical.';
  exit -3;
fi;

for w in $range;
do
  echo "[Worker #$w]" >> "$1.tmp";
done;
cat "$1" >> "$1.tmp";
split -n r/$3 "$1.tmp" "$1.s";
cat "$1.s"* > "$2";
rm "$1."*;

Work perfect.
Now I will see can I adopt it to Win
Thanks!
pepi37 is online now   Reply With Quote
Old 2022-05-10, 22:27   #9
Uncwilly
6809 > 6502
 
Uncwilly's Avatar
 
"""""""""""""""""""
Aug 2003
101ร—103 Posts

101001011110002 Posts
Default

Quote:
Originally Posted by pepi37 View Post
But doesnot work on PRP tests...
I just tried it with some made up PRP lines and it recognized them, sorted them, and attempted to balance them.
Attached Thumbnails
Click image for larger version

Name:	works for me.png
Views:	14
Size:	81.4 KB
ID:	26859  
Uncwilly is offline   Reply With Quote
Old 2022-05-10, 22:53   #10
Batalov
 
Batalov's Avatar
 
"Serge"
Mar 2008
Phi(4,2^7658614+1)/2

27×7×11 Posts
Default

bash
Code:
N=3
for i in `seq 1 $N`; do 
  echo '[Worker #'$i']'; awk 'a++%'$N'=='$i'-1' input.txt
done > output.txt
Batalov is offline   Reply With Quote
Old 2022-05-11, 00:37   #11
paulunderwood
 
paulunderwood's Avatar
 
Sep 2002
Database er0rr

421010 Posts
Default

Here is a perl script. usage: perl myscript.pl input_file worker_count > output_file

Code:
open(fh,"<",$ARGV[0]);
$count=0;
while(<fh>){$count++;}
close(fh);

open(fh,"<",$ARGV[0]);
$per_worker=int($count/$ARGV[1]);
$extras=$count%$ARGV[1];
$worker_blurb="[Worker \#";
$worker=0;
while(<fh>){
        $line=$_;
        $count++;
        if($count>$per_worker){
                $count=0;
                print("\n");
                $worker++;
                print($worker_blurb.$worker."]\n");
                if($extras==0){$per_worker--;}
                $extras--;
        }
        print($line);
}
close(fh);

Last fiddled with by paulunderwood on 2022-05-11 at 01:18
paulunderwood is online now   Reply With Quote
Reply

Thread Tools


Similar Threads
Thread Thread Starter Forum Replies Last Post
BASH script segment won't work in larger script EdH Programming 41 2021-10-15 22:07
CADO-NFS vs GGNFS and other NFS considerations [split from New Factoring Algorithm] Dubslow Factoring 6 2018-04-27 19:56
split a prime95 queue & client installation joblack Information & Answers 1 2009-01-06 08:45
Formula split up JHansen Homework Help 4 2007-11-20 15:02
New theorem to split LL tests(Published) nuggetprime Miscellaneous Math 9 2007-05-23 13:47

All times are UTC. The time now is 20:50.


Sat Jul 2 20:50:54 UTC 2022 up 79 days, 18:52, 0 users, load averages: 1.17, 1.29, 1.22

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2022, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.

โ‰  ยฑ โˆ“ รท ร— ยท โˆ’ โˆš โ€ฐ โŠ— โŠ• โŠ– โŠ˜ โŠ™ โ‰ค โ‰ฅ โ‰ฆ โ‰ง โ‰จ โ‰ฉ โ‰บ โ‰ป โ‰ผ โ‰ฝ โŠ โŠ โŠ‘ โŠ’ ยฒ ยณ ยฐ
โˆ  โˆŸ ยฐ โ‰… ~ โ€– โŸ‚ โซ›
โ‰ก โ‰œ โ‰ˆ โˆ โˆž โ‰ช โ‰ซ โŒŠโŒ‹ โŒˆโŒ‰ โˆ˜ โˆ โˆ โˆ‘ โˆง โˆจ โˆฉ โˆช โจ€ โŠ• โŠ— ๐–• ๐–– ๐–— โŠฒ โŠณ
โˆ… โˆ– โˆ โ†ฆ โ†ฃ โˆฉ โˆช โŠ† โŠ‚ โŠ„ โŠŠ โŠ‡ โŠƒ โŠ… โŠ‹ โŠ– โˆˆ โˆ‰ โˆ‹ โˆŒ โ„• โ„ค โ„š โ„ โ„‚ โ„ต โ„ถ โ„ท โ„ธ ๐“Ÿ
ยฌ โˆจ โˆง โŠ• โ†’ โ† โ‡’ โ‡ โ‡” โˆ€ โˆƒ โˆ„ โˆด โˆต โŠค โŠฅ โŠข โŠจ โซค โŠฃ โ€ฆ โ‹ฏ โ‹ฎ โ‹ฐ โ‹ฑ
โˆซ โˆฌ โˆญ โˆฎ โˆฏ โˆฐ โˆ‡ โˆ† ฮด โˆ‚ โ„ฑ โ„’ โ„“
๐›ข๐›ผ ๐›ฃ๐›ฝ ๐›ค๐›พ ๐›ฅ๐›ฟ ๐›ฆ๐œ€๐œ– ๐›ง๐œ ๐›จ๐œ‚ ๐›ฉ๐œƒ๐œ— ๐›ช๐œ„ ๐›ซ๐œ… ๐›ฌ๐œ† ๐›ญ๐œ‡ ๐›ฎ๐œˆ ๐›ฏ๐œ‰ ๐›ฐ๐œŠ ๐›ฑ๐œ‹ ๐›ฒ๐œŒ ๐›ด๐œŽ๐œ ๐›ต๐œ ๐›ถ๐œ ๐›ท๐œ™๐œ‘ ๐›ธ๐œ’ ๐›น๐œ“ ๐›บ๐œ”