If you have accidentally committed a password or other sensitive data to your Git repository, make sure to remove it before pushing it to GitHub or making it open source. When working with databases or other external systems, it’s incredibly easy to put a password into a configuration file, commit it and forget about it - only for it to be subsequently completely visible to everyone on the internet thereafter.
Thankfully, there are a few ways to remove private data from your Git repos. The most bare-metal approach would be to use the git-filter-branch command which is able to rewrite your Git history whilst applying filters. It’s a very powerful command and therefore has many configuration options and is quite difficult to use directly.
A much easier solution is to use the BFG Repo Cleaner. This is a separate tool which offers a much faster and simpler method of cleaning your repo of private data or very large files.
To make use of this tool, first make sure you have Java installed and available on the PATH - follow the steps in this post to do so. Then download the .jar file from their site.
Find sensitive data within your repository
If you have a password or other credential that you think you may have accidentally committed into your repo, run the following command to find out. This uses the ‘pickaxe’ option of git log to find revisions in which the number of occurences of a search time has changed.
$ git log -S<password>
From the docs:
-S Look for differences that change the number of occurrences of the specified string (i.e. addition/deletion) in a file.
-G: looks for differences whose added or removed line matches the given regexp, as opposed to -S, which “looks for differences that introduce or remove an instance of string”.
–all: searches over all branches and tags
Remove passwords and private data using BFG
Create a file called passwords.txt, in which place all your passwords or sensitive data.
Run the command $ java -jar bfg.jar --replace-text passwords.txt <your-repo>
More documentation and examples, along with how to remove large files from your repo can found on the BFG project page.
Note: This will completely rewrite your Git history, the hashes or all commits will be re-generated.
Visual Studio Code has now replaced Sublime Text as my editor of choice due to it’s speed and first class support for a number of features that were previously only available through extensions (of varying quality and often outdated). One such example is a live preview of a markdown page - something essential when writing blog posts or pretty much any documentation these days.
Thankfully, Visual Studio Code not only provides native support for Markdown, but also a great live preview of the results as you type - all without any extensions. To enable the live preview within a new editor tab, use the keyboard shortcut Ctrl+Shift+V when any Markdown (.md) file is open. This works, but it’s much better to have the live preview of the HTML in a split editor so you can see it as you type. VS Code allows you to do this through the shortcut Ctrl+K V. Alternatively, for either of these options you can go through the Command Palette Ctrl+Shift+P and select either of the ‘Markdown Preview’ options.
As of version 1.9, the markdown preview and editor window both scroll in sync. The preview window now loads in local files and you can also now double click on any element within the preview to jump directly to the corresponding line in the editor tab.
Spelling and grammar checking isn’t currently available within Visual Studio Code out of the box (hopefully this will also be added in a future update). There is however a great extension available on the marketplace which does a great job (when the API it relies on is up).
Note: As of writing file watching is currently not implemented in the WSL so you won’t be able to use jekyll serve --watch. If you are on an insider preview #14942 you will have access to this feature (request page)
Thanks to the awesomeness that is the WSL Windows Subsystem for Linux, you can now install and use a load of tools on Windows just as you would do under Linux. Previously I used to run Jekyll (running under Ruby) solely through a Unix based environment - creating new blog posts through a system of SSH and SFTP. Ruby development on Windows has never really been that great, but now the whole process has got much easier - run everything locally through the WSL layer.
The first step is to install Ruby. Following the instructions from their site:
Your Windows drives are accessible via /mnt using the same drive letters as found in Windows explorer - e.g to access your Windows Documents area via Bash you can cd to:
This works just fine, but it’s a bit long winded when most of your frequently accessed files are located there (plus it looks ugly having such a long path in your prompt all the time). Thankfully, the process gets much simpler through the use of symlinks. Using the following command you can create a symlink pointing directly to your Windows Documents folder:
ln -s /mnt/c/Users/<username>/Documents ~/docs
You can then just do a quick cd ~/docs and end up right in your Windows home directory. The great part is that your prompt will also just show ~/docs instead of the full path via /mnt. I have a number of these symlinks set up, pointing to various places in my Windows drive, and they make the whole experience when using Bash much more fluid.
The Linux kernel often uses available memory for disk caching unless its otherwise needed by a program running on the system. This feature can speed up the system, but the RAM will be marked as used in the top command. It might seem that you are out of memory and need to do something to resolve the problem, when in fact everything is normal and the kernel will simply give programs resources from the disk cache when required. It never takes resources away from programs, just makes use of idle memory sitting in your system. This is such a common question that it even has its own website - http://www.linuxatemyram.com/.
If you really want to clear the disk cache and free the memory, run the following command:
sudo sync && sudo echo 3 | sudo tee /proc/sys/vm/drop_caches
Note: there isn’t much point though, as the kernel will just start using the memory again for disk caching straight after you free it.