Parsl+CWL: Towards Combining the Python and CWL Ecosystems
Uloženo v:
| Vydáno v: | arXiv.org (Dec 11, 2024), p. n/a |
|---|---|
| Hlavní autor: | |
| Další autoři: | , , , , |
| Vydáno: |
Cornell University Library, arXiv.org
|
| Témata: | |
| On-line přístup: | Citation/Abstract Full text outside of ProQuest |
| Tagy: |
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
MARC
| LEADER | 00000nab a2200000uu 4500 | ||
|---|---|---|---|
| 001 | 3143451231 | ||
| 003 | UK-CbPIL | ||
| 022 | |a 2331-8422 | ||
| 035 | |a 3143451231 | ||
| 045 | 0 | |b d20241211 | |
| 100 | 1 | |a Karle, Nishchay | |
| 245 | 1 | |a Parsl+CWL: Towards Combining the Python and CWL Ecosystems | |
| 260 | |b Cornell University Library, arXiv.org |c Dec 11, 2024 | ||
| 513 | |a Working Paper | ||
| 520 | 3 | |a The Common Workflow Language (CWL) is a widely adopted language for defining and sharing computational workflows. It is designed to be independent of the execution engine on which workflows are executed. In this paper, we describe our experiences integrating CWL with Parsl, a Python-based parallel programming library designed to manage execution of workflows across diverse computing environments. We propose a new method that converts CWL CommandLineTool definitions into Parsl apps, enabling Parsl scripts to easily import and use tools represented in CWL. We describe a Parsl runner that is capable of executing a CWL CommandLineTool directly. We also describe a proof-of-concept extension to support inline Python in a CWL workflow definition, enabling seamless use in the Python ecosystem of Parsl. We demonstrate the benefits of this integration by presenting example CWL CommandLineTool definitions that show how they can be used in Parsl, and comparing performance of executing an image processing workflow using the Parsl integration and other CWL runners. | |
| 653 | |a Python | ||
| 653 | |a Parallel programming | ||
| 653 | |a Image processing | ||
| 653 | |a Software | ||
| 653 | |a Workflow | ||
| 700 | 1 | |a Clifford, Ben | |
| 700 | 1 | |a Babuji, Yadu | |
| 700 | 1 | |a Chard, Ryan | |
| 700 | 1 | |a Katz, Daniel S | |
| 700 | 1 | |a Chard, Kyle | |
| 773 | 0 | |t arXiv.org |g (Dec 11, 2024), p. n/a | |
| 786 | 0 | |d ProQuest |t Engineering Database | |
| 856 | 4 | 1 | |3 Citation/Abstract |u https://www.proquest.com/docview/3143451231/abstract/embedded/6A8EOT78XXH2IG52?source=fedsrch |
| 856 | 4 | 0 | |3 Full text outside of ProQuest |u http://arxiv.org/abs/2412.08062 |