Trace:
Differences
This shows you the differences between two versions of the page.
linux:file_management [2018/09/15 11:57] |
linux:file_management [2016/04/14 19:19] (current) onno [Search and Replace in Text Files] |
||
---|---|---|---|
Line 1: | Line 1: | ||
+ | =====File management===== | ||
+ | |||
+ | |||
+ | ==== Using the find command ==== | ||
+ | |||
+ | |||
+ | <code> | ||
+ | find . -name '*httpd*' | ||
+ | </code> | ||
+ | |||
+ | |||
+ | Searches in current directory (.) and deeper for file or directory with httpd in name. | ||
+ | |||
+ | |||
+ | <code> | ||
+ | find /www -ctime -1 | ||
+ | </code> | ||
+ | |||
+ | Searches directory ''**www**'' for files whose status was changed less than 24 hours ago. | ||
+ | |||
+ | |||
+ | <code> | ||
+ | find . -amin +1500 | ||
+ | </code> | ||
+ | |||
+ | Searches current directory for files that were accessed more than 1500 minutes (or 25 hours) ago. | ||
+ | |||
+ | |||
+ | === Excluding a directory from find === | ||
+ | |||
+ | <code> | ||
+ | find /home/ -path '/home/onno' -prune -o -name '*.txt' -ls | ||
+ | 11829831 4 -rw------- 1 ilke ilke 1317 Oct 23 22:46 /home/ilke/.mozilla/firefox/vkuuxfit.default/cookies.txt | ||
+ | </code> | ||
+ | |||
+ | Searches in /home directory and beneath for .txt files, except in ''/home/onno'' directory. | ||
+ | |||
+ | |||
+ | |||
+ | ==== The locate Command ==== | ||
+ | |||
+ | If you already know the filename but not the file's location, use ''locate''. Example: | ||
+ | |||
+ | <code> | ||
+ | # locate libMagickCore.so.1 | ||
+ | /usr/src/ImageMagick-6.4.8-3/magick/.libs/libMagickCore.so.1.0.0 | ||
+ | /usr/src/ImageMagick-6.4.8-3/magick/.libs/libMagickCore.so.1 | ||
+ | /usr/local/lib/libMagickCore.so.1.0.0 | ||
+ | /usr/local/lib/libMagickCore.so.1 | ||
+ | </code> | ||
+ | |||
+ | Apparently, ''locate'' relies on an index (database) which you can update as follows: | ||
+ | |||
+ | <code> | ||
+ | updatedb & | ||
+ | </code> | ||
+ | |||
+ | ==== Other Tricks ==== | ||
+ | |||
+ | <code c> | ||
+ | cat iptables-rules | less | ||
+ | </code> | ||
+ | |||
+ | |||
+ | Shows contents of file //iptables-rules// | ||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | <code c> | ||
+ | chmod 600 iptables-rules | ||
+ | </code> | ||
+ | |||
+ | |||
+ | Change access rights for file //iptables-rules// | ||
+ | |||
+ | |||
+ | |||
+ | Explanation: | ||
+ | |||
+ | 4 - read | ||
+ | |||
+ | 2 - write | ||
+ | |||
+ | 1 - execute | ||
+ | |||
+ | |||
+ | |||
+ | user/owner - group - world | ||
+ | |||
+ | |||
+ | |||
+ | So, for the example given above, only the user/owner may read/write this file. | ||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | ====Copying the contents of a directory==== | ||
+ | |||
+ | |||
+ | Do ''**man cp**'' and ''**man mv**'' to check out the copy and move commands. That said, it took me a while to figure out that the wildcard for files is NOT *.* but simply * under Linux. So, copying the contents of a directory goes like this: | ||
+ | |||
+ | |||
+ | |||
+ | <code c> | ||
+ | [root@1038 public_html]# cp -R ../dummy-3.7.0/* . | ||
+ | </code> | ||
+ | |||
+ | |||
+ | This means: copy the contents (*) of "dummy-.3.7.0 into the current (.) directory. The -R option (recursive) ensures that any subdirectories will be copied along. | ||
+ | |||
+ | |||
+ | |||
+ | |||
+ | ====Creating symbolic links==== | ||
+ | Unix / Linux supports symbolic links, which makes it a very flexible OS. Symbolic links enable you to use a subtree in another subtree without actually copying any directories. Issue this command: | ||
+ | |||
+ | |||
+ | |||
+ | <code c> | ||
+ | ln -s target_dir symlink_name | ||
+ | </code> | ||
+ | |||
+ | |||
+ | Add the full path if necessary. | ||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | ====Creating short text files==== | ||
+ | Linux uses filters such as ''**sort**'' to filter standard input. But you can also use a very simple filter, one which does not filter at all, to view the contents of a file: | ||
+ | |||
+ | |||
+ | |||
+ | <code c> | ||
+ | cat myfile.txt |less | ||
+ | </code> | ||
+ | |||
+ | |||
+ | What's more, you can redirect the standard input using cat to a file: | ||
+ | |||
+ | |||
+ | |||
+ | <code c> | ||
+ | [onno@1038 onno]$ cat > data.txt | ||
+ | This is a test! | ||
+ | </code> | ||
+ | |||
+ | Use Ctrl-D to save the file and get back to the shell prompt. | ||
+ | |||
+ | If you use the ''**>**'' redirect symbol, any data the file may have previously contained, is overwritten. Use ''**>>**'' to append the data to the end of the file: ''**cat >> data.txt**''. | ||
+ | |||
+ | |||
+ | |||
+ | ====Search and Replace in Text Files==== | ||
+ | |||
+ | |||
+ | Use find and sed to search and replace strings in multiple text files. | ||
+ | |||
+ | <code> | ||
+ | find . -name '*.txt' -exec sed -i -e 's/find/replace/g' {} \; | ||
+ | </code> | ||
+ | |||
+ | You can use sed for multiple replacements, too | ||
+ | |||
+ | <code> | ||
+ | find . -name '*.txt' -exec sed -i -e 's/find1/replace1/g' -e 's/find2/replace2/g' -e 's/find3/replace3/g' ... {} \; | ||
+ | </code> | ||
+ | |||
+ | Use control+D to end the input. | ||
+ | |||
+ | Info taken from [[http://snippets.dzone.com/posts/show/4645|this forum]]. | ||
+ | |||
+ | === Find Pattern First === | ||
+ | Take a look at what's going to be replaced first: | ||
+ | |||
+ | <code> | ||
+ | find . -exec grep "find1" '{}' \; -print | ||
+ | </code> | ||
+ | |||
+ | Or, simpler: | ||
+ | |||
+ | <code> | ||
+ | find . | xargs grep 'string' -sl | ||
+ | </code> | ||
+ | NOTE 20160414: didn't work for me anymore. | ||
+ | |||
+ | As explained [[http://www.liamdelahunty.com/tips/linux_find_string_files.php|here]]: | ||
+ | |||
+ | * The -s is for summary and won't display warning messages such as grep: ./directory-name: Is a directory | ||
+ | |||
+ | * The -l is for list, so we get just the filename and not all instances of the match displayed in the results. | ||
+ | |||
+ | **Or, even simpler:** | ||
+ | <code> | ||
+ | grep -Ril "text-to-find-here" . | ||
+ | </code> | ||
+ | |||
+ | * R: Recursive | ||
+ | * i: ignore case | ||
+ | * l: "show the file name, not the result itself" | ||
+ | * .: (the dot) start searching in the current directory | ||
+ | |||
+ | |||
+ | |||
+ | === Rename Files Recursively and Replace Strings inside Files Recursively === | ||
+ | |||
+ | This is a summary of the information provided earlier. | ||
+ | |||
+ | <code> | ||
+ | find . -iname '*valid*' -exec rename 's/valid/mentor/i' {} + | ||
+ | find . -name '*' -exec sed -i -e 's/valid/mentor/g' -e 's/Valid/Mentor/g' {} \; | ||
+ | </code> | ||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | ====Applying patches to files using the patch utility==== | ||
+ | Use the ''**patch**'' utility to apply patches, as contained within ''**.diff**'' files, to other files. Example: | ||
+ | |||
+ | |||
+ | |||
+ | <code> | ||
+ | [root@1038 public_html]# patch -p1 -i ../patches/moodle-1.6-patch.diff | ||
+ | patching file lib/javascript.php | ||
+ | patching file lang/en_utf8/moodle.php | ||
+ | Hunk #1 succeeded at 197 (offset 1 line). | ||
+ | patching file course/format/topics/format.php | ||
+ | patching file course/lib.php | ||
+ | </code> | ||
+ | |||
+ | |||
+ | |||
+ | |||
+ | ====Making backups with tar==== | ||
+ | Use the archiving and compressing utility tar to backup your files. Here's an example where we backup the contents of a webroot dir to a tar file: | ||
+ | |||
+ | |||
+ | |||
+ | <code c> | ||
+ | tar -zcpf tekstenweb20060824.tar public_html | ||
+ | </code> | ||
+ | |||
+ | |||
+ | Explanation: | ||
+ | |||
+ | *z: zip | ||
+ | *c: create | ||
+ | *p: same permissions | ||
+ | *f: make a file (instead of using the standard output) | ||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | ====Renaming Files==== | ||
+ | |||
+ | Simplest: | ||
+ | <code> | ||
+ | mv background_scifi.gif background.gif | ||
+ | </code> | ||
+ | |||
+ | === Bulk Renaming === | ||
+ | I have found the perl program ''**rename**'' to be an excellent tool for this. Its basics are explained on the '[[http://tips.webdesign10.com/how-to-bulk-rename-files-in-linux-in-the-terminal|Webmaster Tips]]' site. | ||
+ | |||
+ | Example: | ||
+ | |||
+ | <code> | ||
+ | rename -n 's/\_scifi//' * | ||
+ | </code> | ||
+ | |||
+ | This replaces the string "scifi" with the "" string (i.e. an empty string), for testing purposes. To do the real thing, simply omit the ''-n'' parameter (or replace it with ''-v'' to see the actual results). | ||
+ | |||
+ | == Recursive Renaming == | ||
+ | Here's another use for ''rename'', recursively: | ||
+ | <code> | ||
+ | find . -iname '*_en.html.erb' -exec rename 's/\_en.html.erb$/\.en.html.erb/i' {} + | ||
+ | </code> | ||
+ | This means: rename all ''*_en.html.erb'' to ''*.en.html.erb'' (replace the underscore by a dot), in the current directory and all subdirectories. | ||
+ | |||
+ | See also [[linux:scripts#recursive_version]] for more examples (replacing spaces in file names). | ||
+ | |||
+ | |||
+ | ====Recursive Copying from an FTP Site==== | ||
+ | |||
+ | How do you copy multiple embedded directories at once from an ftp site? Here's some advice: [[http://forums.devshed.com/ftp-help-113/recursive-mget-with-command-line-ftp-37472.html|forums.devshed.com]]. | ||
+ | |||
+ | It comes down to: ''wget -r ftp://username:password@domain.com/the/path/to/directory'' | ||
+ | |||
+ | If that does not work, try this: | ||
+ | |||
+ | <code> | ||
+ | ncftpget -R -v -u onno -p mysecretpassword the.ftp.host ~/Downloads/destination /the/source/directory | ||
+ | </code> | ||
+ | |||
+ | * -R means recursive | ||
+ | * -v means verbose (show everything) | ||
+ | |||
+ | ====Recursive Copying TO an (S)FTP Site==== | ||
+ | |||
+ | <code> | ||
+ | lftp -c "open -u USER_NAME,PASSWORD sftp://HOST:/TARGET_DIRECTORY ; mirror -R /SOURCE_DIRECTORY" | ||
+ | </code> | ||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | ====Recursive Removal of Files ==== | ||
+ | |||
+ | Combine ''find'' with ''rm'' to recursively delete files. To feed the results of ''find'' to ''rm'', pipe ''find'' to ''xargs''. This is explained on [[http://en.wikipedia.org/wiki/Xargs|wikipedia.org]]. | ||
+ | |||
+ | Example: | ||
+ | |||
+ | <code> | ||
+ | find . -name \*~ | xargs rm | ||
+ | </code> | ||
+ | Delete all files in current directory and subdirectories which end in ''~''. | ||
+ | |||
+ | For files with spaces in their names: | ||
+ | <code> | ||
+ | find . -name "*.mobi" -print0 | xargs -0 rm | ||
+ | </code> | ||
+ | |||
+ | Another example: | ||
+ | |||
+ | <code> | ||
+ | find /home/onno/docs -name \*.bak | xargs rm | ||
+ | </code> | ||
+ | Delete all files in docs and subdirectories which have the file extension ''.bak'' | ||
+ | |||
+ | Alternative: | ||
+ | |||
+ | <code> | ||
+ | find . -name "FILE-TO-FIND"-exec rm -rf {} \; | ||
+ | </code> | ||
+ | |||
+ | |||
+ | ==== Print Files in a Batch ==== | ||
+ | |||
+ | If you want to print all files in a directory, use the cli command ''lpr''. | ||
+ | |||
+ | First you have to find out what your printer is called exactly. This is especially important if you have more than one printer installed. Use the command ''lpstat'': | ||
+ | |||
+ | <code> | ||
+ | lpstat -p -d | ||
+ | printer hp-LaserJet-1300 is idle. enabled since Thu 28 Jul 2011 11:41:06 AM CEST | ||
+ | ready to print | ||
+ | system default destination: hp-LaserJet-1300 | ||
+ | </code> | ||
+ | |||
+ | To find out what options are available for your printer, use the command ''lpoptions'' and specify your printer: | ||
+ | |||
+ | <code> | ||
+ | lpoptions -p hp-LaserJet-1300 -l | ||
+ | PageSize/Media Size: *Letter Legal Executive Statement A4 C5 C6 DL COM10 Monarch | ||
+ | ColorModel/Color Model: *Gray Black | ||
+ | StpColorPrecision/Color Precision: *Normal Best | ||
+ | (... etc ...) | ||
+ | </code> | ||
+ | |||
+ | Finally, to print all pdf documents in the current directory, for instance, issue the ''lpr'' command. WARNING: you will not be asked a confirmation, your printer will get to work immediately. | ||
+ | |||
+ | <code> | ||
+ | lpr -P hp-LaserJet-1300 -o media=A4 *.pdf | ||
+ | </code> | ||
+ | |||
+ | |||
+ | ==== Copy Same File to Multiple Directories ==== | ||
+ | |||
+ | To copy the same file to multiple subdirectories with the same structure, use the ''find'' command: | ||
+ | |||
+ | <code> | ||
+ | cd ~/php | ||
+ | touch delete_me_please.txt | ||
+ | find ./*/public_html/backup/moodle2 -maxdepth 1 -type d -exec cp ~/php/delete_me_please.txt {}/ \; | ||
+ | </code> | ||
+ | |||
+ | This will copy the file ''delete_me_please.txt'' from your php directory into all subdirectories matching the path ''./*/public_html/backup/moodle2''. | ||
+ | |||
+ | This is very nice if you want to copy over a patched file to multiple instances of the same system, say Moodle. | ||
+ | |||
+ | And to delete your test file: | ||
+ | |||
+ | <code> | ||
+ | find ./*/public_html/backup/moodle2/delete_me_please.txt -maxdepth 1 -type f -exec rm {} \; | ||
+ | </code> | ||
+ | |||
+ | |||
+ | |||
+ | |||
+ | ==== Excluding directories from tar ==== | ||
+ | |||
+ | <code> | ||
+ | tar --exclude 'local/soda/.git' --exclude 'local/soda/docs' -zcpf localplugin_soda.20121031.tar.gz local/soda | ||
+ | </code> |
You are here: start » linux » file_management