1 # SPDX-FileCopyrightText: 2021-2022 Ole Tange, http://ole.tange.dk and Free Software and Foundation, Inc.
2 # SPDX-License-Identifier: GFDL-1.3-or-later
3 # SPDX-License-Identifier: CC-BY-SA-4.0
10 parallel, now more precise
14 write a haiku about gnu parallel and the :::+ option
17 Got around to using GNU parallel for the first time from a suggestion by @jdwasmuth ... now I'm wishing I started using this years ago
18 -- Stefan Gavriliuc @GavriliucStefan@twitter
20 I’m a big fan of GNU parallel!
21 -- Scott Cain @scottjcain@twitter
23 gnu parallelが便利すぎてxargsの使い方忘れちゃった
24 -- @mathbbQ@twitter ℕ@凍結解除手続き中
26 Doing some absolutely bonkers "data science" on the command line. GNU parallel + JQ is very powerful, allowing me to parse and 10Gig-JSON-File in under a minute.
27 -- @jeeger@twitter Keter-class developer
29 I've learned a lot during my internship, but getting even slightly more proficient with GNU parallel is probably the most important thing I've learned...
30 -- Elijah Rippeth @terrible_coder@twitter
32 I honestly don't know how I'd survive without @Docker and @GNU_Parallel
33 -- Eric Pauley @EricPauley_@twitter
35 Gnu parallel is indeed slick. I always try to align my data to make it possible to loop over it with a nice for loop added in a call to run the jobs in parallel then becomes super easy. I love the {1..99} syntax in bash.
38 xargs諦めてGNU Parallel使ったら一瞬で問題が解決したので終わり
39 -- aria_moto @moto_aria@twitter
41 Such a life saver of a tool.
44 I used GNU Parallel to run a script locally that did what a big distributed system did, quicker and more reliably. It got to the point where people would just ask me to "run the thing" on my laptop instead of waiting for the cron.
47 parallel is so useful and i use it multiple times daily.
48 -- arrakeen@ycombinator
50 GNU Parallel is one of my favorite utilities of all time.
51 -- paskozdilar@ycombinator
53 Tool, was ich sehr ausgiebig nutze inzwischen und lieben gelernt habe: "GNU parallel"
59 GNU parallel すごいな、シェルスクリプトを make -jX みたいにして並列
61 -- びーる @debugordie@twitter
63 GNU parallel いいよ、大量のデータファイルをスクリプトで処理する人とかにマジおすすめ
64 -- びーる @debugordie@twitter
66 Igual @GnuParallel te puede echar una mano. Te permite pensar y
67 programar en serie pero ejecutar en paralelo usando cores de una
68 máquina o máquinas remotas. Una vez que lo has usando te lamentas no
69 haberlo conocido antes :)
70 -- Juan Sierra Pons @juasiepo
72 If I could only keep 5 GNU utils, parallel would make it to the list
76 Gnu Parallel: installed in every computer i have access to.
77 -- raffaele messuti @atomotic@twitter
79 gnu parallel is a good program
80 -- Pwn A. Day @pwnaday@twitter
82 Deus salve o gnu parallel
83 -- marcos @guv_Tuv@twitter
87 اليوم علمت بأداة تساعد على تشغيل أوامر الصدفة بشكل متوازي حسب عدد الأنوية الموجودة مما يجعلها أداة أساسية في عدة تنفيذ المهام.
88 الأداة اسمها gnu parallel
89 -- @AbuEsra2014@twitter زايد السعيدي
91 GNU parallel is very easy to use and has many features for
92 specialized use cases. It’s a Perl script.
93 -- @harlekyn@twitter uʎʞǝlɹɐɥ
95 GNU parallelという神ツールを発見した
96 -- @schnell1232@twitter
98 How I love GNU parallel
99 -- @dathanvp@twitter Dathan Pattishall
101 GNU Parallelめっちゃ便利で偉い
104 I really liked GNU Parallel http://gnu.org/software/parallel/
105 one of the best tool to execute parallel jobs in the shell
106 -- Luca Molteni @volothamp@twitter
108 Have you heard of our lord and saviour GNU parallel?
109 -- kxyne @Kxyne@twitter
111 Installing GNU parallel is the second thing I do on any new Linux
112 machine, right after uninstalling pulseaudio.
113 -- Andy Cooper @integgroll@twitter
118 parallel is my new kink
119 -- python_noob_001@reddit
121 @GnuParallel the best thing ever and it's not up for debate #EOchat
122 -- Nathan Thomas @DrNASApants@twitter
124 Using [GNU Parallel is] super easy if you use xargs, and it is magic for making things multi-process. Then adding in the ssh magic it can do it is an incredible tool that is completely underutilized.
125 -- Ancients @Ancients@twitter
127 GNU Parallel is one of the most helpful tools I've been using recently, and it's just something like: parallel -j4 'gzip {}' ::: folder/*.csv
128 -- Milton Pividori @miltondp@twitter
130 This is a fantastic tool, and I wish I had upgraded from xargs years ago!
133 GNU Parallel and ripgrep would be your friend here.
134 -- CENAPT @cenaptech@twitter
136 GNU Parallel is great
137 -- Newton's Flaming Laser Sword @swordgoesfwoosh@twitter
139 Try GNU parallel it's awesome, and exactly the thing you are looking for. It allows you to set number of processes running among many things. I use it a lot, and can't recommend it enough.
140 -- mapettheone@reddit
142 It's really quite amazing how powerful and flexible it is
143 -- schwanengesang @tensegrist@twitter
145 Every time I install @ubuntu, one of the first tools I install is
146 @gnuparallel. I love it.
147 -- Necati Demir @ndemir@twitter
149 Today I'm grateful for GNU parallel, especially with the --colsep and
150 --jobs parameters #GiveThanks
151 -- Erin Young @ErinYoun
153 It's not a data migration party until GNU Parallel is involved...
156 -- rrees @rrees@twitter
158 I have gotten a *ton* of mileage out of jq, awk, and GNU parallel,
159 even at multi-GB sizes.
160 -- Eric Wolak @ericthewolak@twitter
162 GNU parallel, which works a little bit like xargs, but has a much more friendly way of handling files with spaces and automatically parallelises calls. This tool has saved me a great deal of coding because it makes it so easy to write a program which does just one part of a task and then run it in parallel with load balancing and a nice progress bar. I cannot recommend this tool enough.
164 https://negfeedback.blogspot.com/2020/05/indispensable-command-line-tools.html
166 GNU parallel all the way!
167 -- David Manouchehri @DaveManouchehri@twitter
169 I found GNU Parallel and it's awesome.
170 -- Teddy Choi @TeddyJChoi@twitter
172 Well anyway, It was blazingly fast and astonished by performance. Guess I'll never use xargs.
173 -- (Not) Akaming @_Akamig@twitter
175 Yeah, GNU parallel is a beast when used accordingly.
178 GNU Parallel is awesome
179 -- Gaurav Jain @gauravjain49@twitter
181 My new favourite command line tool is GNU Parallel. After knowing it for a while I just recently came back to it for a refactor of a script and now I've changed like 4 different huge scripts into tiny parallel scripts that do their job literally 3 to 10 times faster and just [mind blown]
182 -- Izzy @angry_izzy@twitter
184 I love so much @GnuParallel to parallelize my SQL requests on @PostgreSQL
185 -- @rmaziere_85 Romain
187 GNU parallel really changed how I do a lot of data processing stuff
188 -- Brendan Dolan-Gavitt @moyix@twitter
190 Parallel is a life send, what an awesome piece of software.
193 It's really powerful for smallish data processing scripts
194 -- João Veiga @jcsvveiga@twitter
196 It's the MapReduce of our generation!
197 -- Mathias Meyer @roidrage@twitter
199 GNU parallel is a seriously powerful flexible and tool.
200 -- Greg Hurrell @wincent@twitter
202 [socat is] top3 on my "crazy and powerful" list (the other two being
203 gnu parallel and emacs).
204 -- Peter Kjellström @nsccap@twitter
206 GNU/Parallel umm... tempting.
207 -- k-leb k @dcatdemon@twitter
209 Ain't that a dirty hack ;-)
212 Not sure if it counts as an "alt" tool but GNU parallel really took my shell scripting game to the next level.
217 Praise GNU parallel, though. That gets me pretty far.
218 -- Your Obed. Servant, J. B. @Jeffinatorator
221 parallel, with --color flag
222 tasks more vibrant now
225 GNU Parallel absolutely rocks.
226 -- Austin Mordahl@Stackoverflow
228 [GNU Parallel] is the most amazing tool ever invented for bioinformatics!
229 -- Istvan Albert https://www.ialbert.me/
231 If used properly, #gnuparallel actually enables time travel.
232 -- Dr. James Wasmuth @jdwasmuth@twitter
234 reduced our backend test pipelines from 4 to 1.30 hrs. gnu parallel for the win!!!
235 -- Swapnil Sahu @CaffeinatedWryy@twitter
237 Parallel is Good Stuff (tm)
238 -- bloopernova@ycombinator
240 The syntax for GNU Parallel is so slick that I often use it just to make my script read nicer, and the parallelism is a cherry on top.
243 Parallel has been (and still is) super useful and simple tool for speeding up all kinds of shell tasks during my career.
244 -- ValtteriL@ycombinator
246 It's amazing how fast you can get with bash pipelines and GNU Parallel.
247 -- Eric Pauley @EricPauley_
249 Immensely useful which I am forever grateful that it exists.
250 -- AlexDragusin@ycombinator
252 My favorite software, ever. Keep the good work.
253 -- Federico Alves @federicoalves@twitter
255 I also prefer gnu parallel. Mainly because it makes embarrassingly
256 parallel tasks embarrassingly easy to run on the command line.
257 -- Vincent D. Warmerdam @fishnets88@twitter
259 I'm going to make GNU parallel my new superpower this year.
260 -- Christian G. Warden @xn@twitter
262 My favorite man page is that of GNU parallel.
263 -- Jeroen Janssens @jeroenhjanssens@twitter
266 -- @butagannen@twitter 豚顔面
268 GNU parallel is a severely underrated tool for just spawning a
269 shitload of tasks. I use it every day and fucking love it.
270 -- Jane @serialexpjane@twitter
272 Take a look at GNU Parallel. It’s magnificent.
273 -- Siira@ycombinator.com
275 Safe to say, @GnuParallel was a life changer during my PhD! It helped
276 me optimise so many of my tasks and analyses.
277 -- Parice Brandies @PariceBrandies@twitter
279 We use gnu parallel now - and happier for it.
280 -- Ben Davies @benjamindavies@twitter
282 GNU Parallel makes my life so much easier.
283 I'm glad I don't have to implement multi-threaded Python scripts on the regular.
284 -- Fredrick Brennan @fr_brennan@twitter
286 If you work with lots of files at once
287 Take a good look at GNU parallel
288 Change your life for the better
289 -- French @notareverser@twitter
291 GNU Parallel is your friend.
292 Can shorten that time by X cores.
293 -- iRODS @irods@twitter
295 GNU Parallel is my new favorite thing
296 -- Will Tejeda @thewilltejeda
298 GNU Parallel is a priceless timesaver
299 -- June "Wakalix" @TWakalix@twitter
301 I think many people would be surprised to learn that GNU parallel is
302 "just" a single Perl script.
303 -- Peter Menzel @ptr_menzel@twitter
305 GNU Parallel is my single favourite tool for batch processing data
306 from the command line.
307 -- Jeff Wintersinger @jwintersinger
309 GNU parallel should be taught in class, it is one of the best tools
310 to run grids of experiments
311 -- no love deep learning @tetraduzione@twitter
313 I get a weird sense of satisfaction every single time I see the
314 lovely logo of #GNU Parallel (plus, what an underrated piece of
316 -- Emre Sevinç @EmreSevinc@twitter
318 My favorite man page is that of GNU parallel.
319 -- Jeroen Janssens @jeroenhjanssens@twitter
321 Gnu parallel is also awesome, fwiw.
322 -- Rogan Dawes @RoganDawes@twitter
324 With multicore systems everywhere GNU Parallel is a must have tool.
325 -- Neil H. Watson @neil_h_watson@twitter
327 Who needs spark when GNU Parallel exists
328 -- MatthijsB @MatthijsBrs@twitter
330 GNU Parallel: dead simple process-level parallelization of ad hoc
331 tasks. Write for a chunk, let gnu manage the splitting, permutations
332 and pool concurrency.
333 -- Nick Ursa @nickursa@twitter
335 I wish more command line software had example pages as robust as GNU Parallel
336 -- Lucidbeaming @lucidbeaming
338 GNU parallel is straight up incredible.
339 -- Ben Johnson @biobenkj@twtter
341 GNU parallel has helped me kill a Hadoop cluster before.
342 -- Travis Campbell @hcoyote@twitter
344 Parallel is the BEST command.
345 -- Nick @NickInfoSec@twitter
347 [L]earning about parallel was amazing for me, it gives us many beautiful solutions.
348 -- SergioAraujo@stackoverflow
350 I've said it before: The command line program GNU Parallel is a godsend.
351 -- Jo Chr. Oterhals @oterhals
353 IMHO, SQLite and GNU Parallel are among the world's great software.
356 It is, beyond absolutely any doubt whatsoever, the single most
357 important tool I use in making me a productive bioinformatician.
358 -- A-N-Other@reddit.com
360 It is SUPER easy to speed up jobs from the command line w/ GNU parallel.
361 -- B3n @B3njaminHimes@twitter
363 I want to make a shout-out for @GnuParallel, it's a work of beauty and power
364 -- Cristian Consonni @CristianCantoro
366 Amazingly useful script!
369 Y'all need some GNU parallel in your lives
370 -- ChaKu @ChaiLovesChai@twitter
373 There are so many things to love about GNU parallel. You could honestly teach a whole parallel computing course with it and never have to leave it for a real language.
374 -- Aubrey Bailey @DNAvinci@twitter
377 With GNU Parallel you sure can!
378 I like getting things done
380 --Kyle Lady @kylelady@twitter
382 Ok! GNU Parallel is one of the best things out there. Almost as good as vanilla ice cream.
383 -- @coffe@mastodon.art
385 HOLY STUFF I LOVE GNU PARALLEL
386 -- Nick (but spookier) @NickInfoSec@twitter
389 What this country needs is a president with a strong understanding of GNU Parallel, Awk, Sed, Grep, and Diff
390 -- Gregg Housh @GreggHoush
392 I know I'm late to the party but GNU Parallel is truly amazing!
393 -- Sam Diaz-Munoz @sociovirology
395 GNU parallel is a thing of magic.
396 -- Josh Meyer @joshmeyerphd@twitter
398 I've been using GNU Parallel very much and effectively lately.
399 Such an easy way to get huge speed-ups with my simple bash/Perl/Python
400 programs -- parallelize them!
401 -- Ken Youens-Clark @kycl4rk@twitter
404 export them more easily
409 GNU Parallel is cool.
410 Concurrency, but in the Unix-philosophy style,
411 without the Enterprise wankeriness.
412 -- NickM bokkiedog@twitter
414 An ode to GNU parallel
415 An ode to GNU parallel
416 An ode to GNU parallel
417 An ode to GNU parallel
418 An ode to GNU parallel
419 An ode to GNU parallel
420 -- Adam Stuckert PoisonEcology@twitter
423 limit jobs dynamic'ly
428 touches files on disk no more
432 Always test your script
438 used to be a static string
443 all data to all the jobs
447 Speed is good for you
448 Multi speed is better still
453 many job outputs at once:
457 Pipes are fast and good.
458 Use them in your programs, too.
464 niceload dash dash net
473 Functions, vars and other stuff
478 SQL would help a lot.
482 Dash dash pipe too slow?
483 Dash dash pipepart is your friend.
493 You should then learn dash dash pipe.
498 everything is quick and fast
503 faster run than single jobs
508 Doing nothing. I say no!
513 Multiple can run at once.
518 Idle cores is not your thing.
523 Instant cluster needed now?
527 xargs' space and quote
528 headache causing behaviour.
533 Run jobs using Parallel.
538 Want a simpler syntax now?
542 Shellshock pain hits us.
543 Dash dash env is affected.
548 quoting is still a black art
552 bash for loops vanquished
553 jobs fan out over network
557 code fork headache blues?
558 option P is your new friend
563 inflexible replacements?
564 Use Perl expressions.