Import modules must target the files they want to process

If import modules target the file they want to process they would avoid processing unwanted files.

was : TBX: x.x, don't process hidden files of a source directory :

Import modules must avoid processing hidden files (Java point of view, tilde files…)

Import modules manipulating XML files are particularly concerned because hidden files may interrupt the import process : https://groupes.renater.fr/wiki/txm-info/public/environnement\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\_import\\\\\\\\\\\\\\\\#bug\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\_import\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\_xmlw\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\_csv\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\_de\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\_txm\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\_075\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\_fichiers\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\_tild\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\_et\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\_caches

see import module input specifications (file, files, directory, directories, URL, etc.)

(from redmine: issue id 824, created on 2014/05/21 by Matthieu Decorde)

• Relations:
  • relates #838

Solution

Finalize and replace new File().listFiles() method with the org.txm.utils.FileUtils.listFiles(dir, ..., exts, ...) method in all the Groovy scripts

Validation test

• run all import modules
• try adding hidden files, directories and *.txt files with a XTZ import

added import status::in progress type::bug + 1 deleted label

changed milestone to %TXM 0.8.4

In TXM 0.7.7, the XML/w+CSV import module tries to use the ‘metadata.csv~’ file as a source file.

Solution

Ignore ‘*~’ files in the source directory.

(from redmine: written on 2015/09/20 by Serge Heiden)

I move it to 0.7.9, some points are not done yet.

(from redmine: written on 2015/11/13 by Matthieu Decorde)

mentioned in issue #838

changed milestone to %TXM 0.8.4

changed title from TBX: x.x, don't process hidden files of a source directory to Import modules must target the files they want to process

changed the description

removed status::in progress label

changed the description

added status::à tester label

closed

removed status::à tester label

What solution has been implemented ?

changed the description