Implemented XML Downloads/Export via API for Future Automation in Mermeid #263 #267
Conversation
|
Hi @imranasif87 , that's is really great work and worked out of the box for me! I'm only a little bit hesitant about the endpoint "download-xml" since I do think what it actually implements is the missing "list" functionality of our CRUD module. Hence, my proposal would be to I'm happy to do these backend changes if you do not have the time?! |
resources/js/download.js
Outdated
| progressLabel.textContent = `Downloading file ${i + 1} of ${totalFiles}`; | ||
| currentFileLabel.textContent = `Currently downloading: ${file}`; | ||
|
|
||
| let resp = await fetch(`http://localhost:8080/data/${encodeURIComponent(file)}`) |
There was a problem hiding this comment.
the server name etc. should not be hard coded.
Can you get this information from some existing resource or should we add this information to the response of the backend endpoint?
|
Hi @peterstadler, I've updated the solution based on your feedback—please review it when you get a chance. Additionally, I have a suggestion for enhancing MerMEId: since users can now download all the files, we could consider supporting a re-upload mechanism. Currently, if a user updates the MerMEId version and re-runs or rebuilds the image without a cache, all previously stored files are lost. To address this, we could allow users to upload a ZIP archive containing all the XML files after logging in. The front-end could then extract and back-end has methods like crud:create into the eXist-db. This functionality could also support automated uploads via local bash scripts in the future. Just an idea for discussion—happy to hear your thoughts on this. |
resources/js/download.js
Outdated
| for (let i = 0; i < totalFiles; i++) { | ||
| const file = filteredFiles[i]; | ||
|
|
||
| if (!file.endsWith(".xml")) |
There was a problem hiding this comment.
That's worth a discussion ;)
a) I have also seen the file extension ".mei" which should not be excluded here.
b) what about letting the backend decide what a MerMEId data file is?
In general, it's probably not that easy. You would probably need to look into the file to tell whether it's an MEI file or not – and that could get get expensive for large databases.
There was a problem hiding this comment.
@peterstadler, To address this, I added a code snippet that filters and returns only the files where the root element is . While this approach may not be suitable for very large data files, it performs efficiently for reasonably sized datasets. Moving forward, we should consider implementing indexing or caching to improve scalability.
- Preprocess the collection to tag or index files with roots
- Store valid MEI filenames in a metadata file or use custom indexes
- Avoids runtime parsing for every request
|
Many thanks for all the great work and your patience @imranasif87 ! |
I replaced the link button with an image button and added a tooltip to clarify its function.