Differences

This shows you the differences between two versions of the page.

Link to this comparison view

linux:file_management [2018/09/15 11:57]
linux:file_management [2016/04/14 19:19] (current)
onno [Search and Replace in Text Files]
Line 1: Line 1:
 +=====File management=====
  
 +
 +
 +==== Using the find command ====
 +
 +
 +<code>
 +find . -name '*httpd*'
 +</code>
 +
 +
 +Searches in current directory (.) and deeper for file or directory with httpd in name.
 +
 +
 +<code>
 +find /www -ctime -1
 +</code>
 +
 +Searches directory ''**www**'' for files whose status was changed less than 24 hours ago.
 +
 +
 +<code>
 +find . -amin +1500
 +</code>
 +
 +Searches current directory for files that were accessed more than 1500 minutes (or 25 hours) ago.
 +
 +
 +=== Excluding a directory from find ===
 +
 +<code>
 + find /home/ -path '/home/onno' -prune -o -name '*.txt' -ls
 + 11829831    4 -rw-------   1 ilke     ilke         1317 Oct 23 22:46 /home/ilke/.mozilla/firefox/vkuuxfit.default/cookies.txt
 +</code>
 +
 +Searches in /home directory and beneath for .txt files, except in ''/home/onno'' directory.
 +
 +
 +
 +==== The locate Command ====
 +
 +If you already know the filename but not the file's location, use ''locate''. Example:
 +
 +<code>
 +# locate libMagickCore.so.1
 +/usr/src/ImageMagick-6.4.8-3/magick/.libs/libMagickCore.so.1.0.0
 +/usr/src/ImageMagick-6.4.8-3/magick/.libs/libMagickCore.so.1
 +/usr/local/lib/libMagickCore.so.1.0.0
 +/usr/local/lib/libMagickCore.so.1
 +</code>
 +
 +Apparently, ''locate'' relies on an index (database) which you can update as follows:
 +
 +<code>
 +updatedb &
 +</code>
 +
 +==== Other Tricks ====
 +
 +<code c>
 +cat iptables-rules | less
 +</code>
 +
 +
 +Shows contents of file //iptables-rules// 
 +
 +
 +
 +
 +
 +<code c>
 +chmod 600 iptables-rules 
 +</code>
 +
 +
 +Change access rights for file //iptables-rules//
 +
 +
 +
 +Explanation:
 +
 +4 - read
 +
 +2 - write
 +
 +1 - execute
 +
 +
 +
 +user/owner - group - world
 +
 +
 +
 +So, for the example given above, only the user/owner may read/write this file. 
 +
 +
 +
 +
 +
 +
 +====Copying the contents of a directory====
 +
 +
 +Do ''**man cp**'' and ''**man mv**'' to check out the copy and move commands. That said, it took me a while to figure out that the wildcard for files is NOT *.* but simply * under Linux. So, copying the contents of a directory goes like this:
 +
 +
 +
 +<code c>
 +[root@1038 public_html]# cp -R ../dummy-3.7.0/* .
 +</code>
 +
 +
 +This means: copy the contents (*) of "dummy-.3.7.0 into the current (.) directory. The -R option (recursive) ensures that any subdirectories will be copied along.
 +
 +
 +
 +
 +====Creating symbolic links====
 +Unix / Linux supports symbolic links, which makes it a very flexible OS. Symbolic links enable you to use a subtree in another subtree without actually copying any directories. Issue this command:
 +
 +
 +
 +<code c>
 +ln -s target_dir symlink_name
 +</code>
 +
 +
 +Add the full path if necessary.
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +====Creating short text files====
 +Linux uses filters such as ''**sort**'' to filter standard input. But you can also use a very simple filter, one which does not filter at all, to view the contents of a file:
 +
 +
 +
 +<code c>
 +cat myfile.txt |less
 +</code>
 +
 +
 +What's more, you can redirect the standard input using cat to a file:
 +
 +
 +
 +<code c>
 +[onno@1038 onno]$ cat > data.txt
 +This is a test!
 +</code>
 +
 +Use Ctrl-D to save the file and get back to the shell prompt.
 +
 +If you use the ''**>**'' redirect symbol, any data the file may have previously contained, is overwritten. Use  ''**>>**'' to append the data to the end of the file: ''**cat >> data.txt**''.
 +
 +
 +
 +====Search and Replace in Text Files====
 +
 +
 +Use find and sed to search and replace strings in multiple text files.
 +
 +<code>
 +find . -name '*.txt' -exec sed -i -e 's/find/replace/g' {} \;
 +</code>
 +
 +You can use sed for multiple replacements, too
 +
 +<code>
 +find . -name '*.txt' -exec sed -i -e 's/find1/replace1/g' -e 's/find2/replace2/g' -e 's/find3/replace3/g' ... {} \;
 +</code>
 +
 +Use control+D to end the input.
 +
 +Info taken from [[http://snippets.dzone.com/posts/show/4645|this forum]].
 +
 +=== Find Pattern First ===
 +Take a look at what's going to be replaced first:
 +
 +<code>
 +find . -exec grep "find1" '{}' \; -print
 +</code>
 +
 +Or, simpler:
 +
 +<code>
 +find . | xargs grep 'string' -sl
 +</code>
 +NOTE 20160414: didn't work for me anymore.
 +
 +As explained [[http://www.liamdelahunty.com/tips/linux_find_string_files.php|here]]:
 +
 +  * The -s is for summary and won't display warning messages such as grep: ./directory-name: Is a directory
 +
 +  * The -l is for list, so we get just the filename and not all instances of the match displayed in the results.
 +
 +**Or, even simpler:**
 +<code>
 +grep -Ril "text-to-find-here" .
 +</code>
 +
 +  * R: Recursive
 +  * i: ignore case
 +  * l: "show the file name, not the result itself"
 +  * .: (the dot) start searching in the current directory
 +
 +
 +
 +=== Rename Files Recursively and Replace Strings inside Files Recursively ===
 +
 +This is a summary of the information provided earlier.
 +
 +<code>
 +find . -iname '*valid*' -exec rename 's/valid/mentor/i' {} +
 +find . -name '*' -exec sed -i -e 's/valid/mentor/g' -e 's/Valid/Mentor/g' {} \;
 +</code>
 +
 +
 +
 +
 +
 +
 +====Applying patches to files using the patch utility====
 +Use the ''**patch**'' utility to apply patches, as contained within ''**.diff**'' files, to other files. Example:
 +
 +
 +
 +<code>
 +[root@1038 public_html]# patch -p1 -i ../patches/moodle-1.6-patch.diff
 +patching file lib/javascript.php
 +patching file lang/en_utf8/moodle.php
 +Hunk #1 succeeded at 197 (offset 1 line).
 +patching file course/format/topics/format.php
 +patching file course/lib.php
 +</code>
 +
 +
 +
 +
 +====Making backups with tar====
 +Use the archiving and compressing utility tar to backup your files. Here's an example where we backup the contents of a webroot dir to a tar file:
 +
 +
 +
 +<code c>
 +tar -zcpf tekstenweb20060824.tar public_html
 +</code>
 +
 +
 +Explanation:
 +
 +  *z: zip
 +  *c: create
 +  *p: same permissions
 +  *f: make a file (instead of using the standard output)
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +====Renaming Files====
 +
 +Simplest:
 +<code>
 +mv background_scifi.gif background.gif
 +</code>
 +
 +=== Bulk Renaming ===
 +I have found the perl program ''**rename**'' to be an excellent tool for this. Its basics are explained on the '[[http://tips.webdesign10.com/how-to-bulk-rename-files-in-linux-in-the-terminal|Webmaster Tips]]' site.
 +
 +Example:
 +
 +<code>
 +rename -n 's/\_scifi//' *
 +</code>
 +
 +This replaces the string "scifi" with the "" string (i.e. an empty string), for testing purposes. To do the real thing, simply omit the  ''-n'' parameter (or replace it with ''-v'' to see the actual results).
 +
 +== Recursive Renaming ==
 +Here's another use for ''rename'', recursively:
 +<code>
 +find . -iname '*_en.html.erb' -exec rename 's/\_en.html.erb$/\.en.html.erb/i' {} +
 +</code>
 +This means: rename all ''*_en.html.erb'' to ''*.en.html.erb'' (replace the underscore by a dot), in the current directory and all subdirectories.
 +
 +See also [[linux:scripts#recursive_version]] for more examples (replacing spaces in file names).
 +
 +
 +====Recursive Copying from an FTP Site====
 +
 +How do you copy multiple embedded directories at once from an ftp site? Here's some advice: [[http://forums.devshed.com/ftp-help-113/recursive-mget-with-command-line-ftp-37472.html|forums.devshed.com]].
 +
 +It comes down to: ''wget -r ftp://username:password@domain.com/the/path/to/directory''
 +
 +If that does not work, try this:
 +
 +<code>
 +ncftpget -R -v -u onno -p mysecretpassword the.ftp.host ~/Downloads/destination /the/source/directory
 +</code>
 +
 +  * -R means recursive
 +  * -v means verbose (show everything)
 +
 +====Recursive Copying TO an (S)FTP Site====
 +
 +<code>
 +lftp -c "open -u USER_NAME,PASSWORD sftp://HOST:/TARGET_DIRECTORY ; mirror -R /SOURCE_DIRECTORY"
 +</code>
 +
 +
 +
 +
 +
 +====Recursive Removal of Files ====
 +
 +Combine ''find'' with ''rm'' to recursively delete files. To feed the results of ''find'' to ''rm'', pipe ''find'' to ''xargs''. This is explained on [[http://en.wikipedia.org/wiki/Xargs|wikipedia.org]].
 +
 +Example:
 +
 +<code>
 +find . -name \*~ | xargs rm
 +</code>
 +Delete all files in current directory and subdirectories which end in ''~''.
 +
 +For files with spaces in their names:
 +<code>
 +find . -name "*.mobi" -print0 | xargs -0 rm
 +</code>
 +
 +Another example:
 +
 +<code>
 +find /home/onno/docs -name \*.bak | xargs rm
 +</code>
 +Delete all files in docs and subdirectories which have the file extension ''.bak''
 +
 +Alternative:
 +
 +<code>
 +find . -name "FILE-TO-FIND"-exec rm -rf {} \;
 +</code>
 +
 +
 +==== Print Files in a Batch ====
 +
 +If you want to print all files in a directory, use the cli command ''lpr''.
 +
 +First you have to find out what your printer is called exactly. This is especially important if you have more than one printer installed. Use the command ''lpstat'':
 +
 +<code>
 +lpstat -p -d
 +printer hp-LaserJet-1300 is idle.  enabled since Thu 28 Jul 2011 11:41:06 AM CEST
 + ready to print
 +system default destination: hp-LaserJet-1300
 +</code>
 +
 +To find out what options are available for your printer, use the command ''lpoptions'' and specify your printer:
 +
 +<code>
 +lpoptions -p hp-LaserJet-1300 -l
 +PageSize/Media Size: *Letter Legal Executive Statement A4 C5 C6 DL COM10 Monarch
 +ColorModel/Color Model: *Gray Black
 +StpColorPrecision/Color Precision: *Normal Best
 +(... etc ...)
 +</code>
 +
 +Finally, to print all pdf documents in the current directory, for instance, issue the ''lpr'' command. WARNING: you will not be asked a confirmation, your printer will get to work immediately.
 +
 +<code>
 +lpr -P hp-LaserJet-1300 -o media=A4 *.pdf
 +</code>
 +
 +
 +==== Copy Same File to Multiple Directories ====
 +
 +To copy the same file to multiple subdirectories with the same structure, use the ''find'' command:
 +
 +<code>
 +cd ~/php
 +touch delete_me_please.txt
 +find ./*/public_html/backup/moodle2 -maxdepth 1 -type d -exec cp ~/php/delete_me_please.txt {}/ \;
 +</code>
 +
 +This will copy the file ''delete_me_please.txt'' from your php directory into all subdirectories matching the path ''./*/public_html/backup/moodle2''.
 +
 +This is very nice if you want to copy over a patched file to multiple instances of the same system, say Moodle.
 +
 +And to delete your test file:
 +
 +<code>
 +find ./*/public_html/backup/moodle2/delete_me_please.txt -maxdepth 1 -type f -exec rm {} \;
 +</code>
 +
 +
 +
 +
 +==== Excluding directories from tar ====
 +
 +<code>
 +tar --exclude 'local/soda/.git' --exclude 'local/soda/docs' -zcpf localplugin_soda.20121031.tar.gz local/soda
 +</code>

Personal Tools