OpenMediaVault NAS Setup with Docker Applications
Networking
My 10 year old NAS finally died after 10 years. It ran Ubuntu, with a bunch of apps and services installed via the OS's package manager, and some installed and compiled from source. I used the LTS version of Ubuntu for longer term support, and I actually did full OS upgrades once or twice. However, I think I stopped since version 16 or so. I remember after doing full upgrades, certain apps or setup would break (packages removed, dependencies removed or are no longer compatible, etc.).
Force new WAN IP from ISP with new WAN MAC address on router
Internet
Networking
I recently wanted my ISP (Spectrum Internet, formerly Time Warner Cable) to issue me a new IP address. I called the provider and they informed me they couldn't do it manually. The only way for me to get a new IP address is to wait until the routine planned scheduled date (in a month) for a new IP to be issued, or to get a new cable modem (ISP would assign a new IP to the new MAC address). I opted for the latter because they offered me a free modem as part of my plan, and because I wanted this done ASAP. However, the ISP still issued me the same WAN IP to my router when I plugged in the new modem. I next tried plugging my laptop directly into the modem using an ethernet cable. The ISP did issue me a new IP address. This shows that the ISP issues an IP address to the device plugged into the modem, not to the modem.
Note on using T-SQL via R and ODBC
R
Reminder to self: when trying to query data into R using an existing SQL script that uses T-SQL to access data in Microsoft SQL Server, add
SET NOCOUNT ON;
to the beginning of the script, then add SET NOCOUNT OFF
before the last select statement (the query returning the data of interest). If this is not done, an empty is returned instead.
Estimating standard errors using the Delta Method in R
R
Statistics
I just discovered the msm package in R that offers the
deltamethod
function for calculating the standard error of a transformation of estimated regression coefficients. A tutorial could be found here. I had to use it recently to calculate the standard error for a difference in proportions when logistic regression model was used to estimate the log-odds of an event:
Secure access to home IP cameras over the web with SSL
Linux
Networking
IP cameras are cheap these days. I have a few Foscam FI8910W cameras at home as part of my home security system. A friend recently gave me a Foscam C1 to be used as a baby monitor. I try to avoid using the manufacturer's mobile device apps for viewing the video feeds because 1) I don't want my private information (video, IP address, credentials, etc) exchanged or leaked to the manufacturer and 2) I don't want to have to use multiple apps if I buy multiple cameras from different manufacturers. I currently use tinyCam Monitor PRO on my Android device to view the video feed both on the home network and while I'm away. It works very well, is customizable, and supports many different manufacturers.
Using Vagrant to scale data science projects with Google Cloud Compute
Linux
Statistics
When I was in graduate school, I made heavy use of the school's computing infrastructure for my research by scheduling many simulation jobs, utilizing multiple (if not all) compute nodes in parallel using Grid Engine. In my professional life, I've always pushed to have a computing environment dedicated for research and analysis. This was typically in the form of a Linux or UNIX server with many CPU cores (64 one) and as much ram as I could get. The difficulty in getting the approval to have such an environment depended on the company's culture, so YMMV. The beauty of the work setup over the graduate school setup is that a job scheduler was never needed as the number of concurrent users vying for compute cycles are drastically less at work. When building a computing environment, I always try to build the beefiest server possible (translation: that I could get approval for) because I never want to run into a project that the server couldn't handle (eg, loading a very large data set into memory with R). However, it's hard to future-proof all projects completely so the line had to be drawn somewhere and thus the number of CPU cores and memory had a limit.
Google Voice using SIP via Simonics
Linux
Google Voice ended third-party support for XMPP clients in May 2014. My Asterisk auto-attendant remained functional into 2015. However it's finally broken for me: when people call in, or when people don't call in, some digits get pressed randomly in Asterisk (probably because of some negotiation between Asterisk and GV via XMPP) and calls are forwarded sporadically throughout the day (what I call phantom calls). Is there a way to still leverage Google Voice as my auto-attendant? I found this post talking about Simon Telephonics offering a GV gateway using SIP. There is a $5 setup fee. I paid it and now am able to continue using it with Asterisk as my auto-attendant. It also works with any SIP phone or SIP app (eg, CSipSimple). I like this solution because I still get free calls using Google Voice without having to use another third-party to pay for my calls.
Automatically specify line break options with termstr as CRLF or LF in SAS when importing data
statistics
It could be annoying when dealing with data from multiple platforms: Windows uses the carriage return (CR) and line feed (LF) to indicate a new line, UNIX uses LF, and Mac uses CR. Most companies have SAS running on a UNIX/Linux server, and it's hard to tell which characters indicate a new line without going to a terminal to inspect the file.
Compile R 3.2.2 on AIX 6.1
Linux
R
Statistics
Here are my notes compiling 64-bit R 3.2.2 on AIX 6.1. As a pre-requisite, read the AIX notes from R-admin. Like the notes, I had GCC installed from here by our admin, along with many other pre-requisites. These were installed prior to compiling R. Note that you could grab newer versions of each package by going to
http://www.oss4aix.org/download/RPMS/
(needed for R-dev).
Repair line breaks within a field of a delimited file
Statistics
Linux
Sometimes some people generate delimited files with line break characters (carriage return and/or line feed) inside a field without quoting. I previously wrote about the case when the problematic fields are quoted. I also wrote about using non-ascii characters as field and new record indicators to avoid clashes.
Calculate the weighted Gini coefficient or AUC in R
R
statistics
This post on Kaggle provides R code for calculating the Gini for assessing a prediction rule, and this post provides R code for the weighted version (think exposure for frequency and claim count for severity in non-life insurance modeling). Note that the weighted version is not well-defined when there are ties in the predictions and where the corresponding weights vary because different Lorentz curve (gains chart) could be drawn for different orderings of the observations; see this post for an explanation and some examples.
Make a printer wireless using a router with USB running OpenWRT
Networking
Many recent printers have wifi capability built in for wireless printing. Older printers or even some recent printers do not have this feature, but one could purchase a wireless adapter to turn the printer wireless. The adapters aren't cheap, and a search for a cheap adapter led me to configuring the TP-Link WR-703N with OpenWRT as an affordable alternative (plug printer into router with a usb cable and print to router via a usb print server).
Optimized R and Python: standard BLAS vs. ATLAS vs. OpenBLAS vs. MKL
Linux
R
Python
Revolution Analytics recently released Revolution Open R, a downstream version of R built using Intel's Math Kernel Library (MKL). The post mentions that comparable improvements are observed on Mac OS X where the ATLAS blas library is used. A reader also expressed his hesitation in the Comments section for a lack of a comparison with ATLAS and OpenBLAS. This concept of using a different version of BLAS is documented in the R Administration manual, and has been compared in the past here and here. Now, as an avid R user, I should be using a more optimal version of R if it exists and is easy to obtain (install/compile), especially if the improvements are up to 40% as reported by the Domino Data Lab. I decided to follow the framework set out by this post to compare timings for the different versions of R on a
t2.micro
instance on Amazon EC2 running Ubuntu 14.04.
Build R on AIX
Linux
R
http://cran.r-project.org/doc/manuals/r-release/R-admin.html#AIX CC
"gcc -maix64 -pthread" CXX
"g++ -maix64 -pthread" FC="gfortran -maix64 -pthread" F77="gfortran -maix64 -pthread" CFLAGS="-O2 -g -mcpu=power6" FFLAGS="-O2 -g -mcpu=power6" FCFLAGS="-O2 -g -mcpu=power6" C:;C: Files;C: Files.0;C: Files.1;C:;C:;%SystemRoot%;%SystemRoot%;%SystemRoot%;%SYSTEMROOT%1̌.0\;C: Files (x86) Access Client\
Change delimiter in a csv file and remove line breaks in fields
Linux
Python
Coding
I wrote a script to convert delimiters in CSV files, eg, commas to pipes. I prefer pipe-delimited files because the the pipe-delimiter (|) will not clash data in the different fields 99.999% of the time. I also added the option to convert newline () and carriage return ()̊ characters in the data fields to spaces. This comes in handy when I use
PROC IMPORT
in SAS as line breaks cause it to choke.
Issues with https proxy in Python via suds and urllib2
Linux
Python
Coding
I recently had the need to access a SOAP API to obtain some data. SOAP works by posting an xml file to a site url in a format defined by the API's schema. The API then returns data, also in a form of an xml file. Based on this post, I figured suds was the easiest way to utilize Python to access the API so I could sequentially (and hence, parallelize) query data repeatedly.
suds
did turn out to be relatively easy to use:
Package management in R and Python at work without root and behind firewall
R
Coding
Linux
My current job has strict security measures (referring to root access on a Linux server and the inability to access outside the company's network), so it can be difficult in getting the tools necessary for my work, namely R packages on CRAN and Python packages via
pip
.
Upgrading Ubuntu 12.04 to 14.04 breaks encrypted LVM
Linux
My laptop runs Ubuntu and is fully encrypted (since version 10.04). Upgrade from 10.04 to 12.04 was smooth in the sense that my system booted fine, asking for the passphrase to unlock the LVM. However, when I upgraded from 12.04 to 14.04, things broke and my laptop no longer booted properly as the LVM never got encrypted. I had to do the following to get my laptop working again (after many rounds of trial and error):
Parental control on home network
Networking
Internet
I recently looked into ways to block content on the home network. To protect the entire network, it seems like the filter should be placed on the router. This article on Lifehacker lists a few popular methods. I think using OpenDNS to filter is easy enough to get started. However, it's quite easy to configure your connected computer to use a different DNS provider. However, one could set a static DNS on their tomato router.
Output data to Excel for reproducible post-hoc analysis or visualization
R
SAS
As much as I like to analyze and visualize data in R, I sometimes have the need to export results/data into Excel for my business partners or myself to consume in Excel or Powerpoint (eg, create custom/edit-able bar charts with various graphical overlays in a powerpoint slide). As I've been using the XLConnect package to read
xls/xlsx
files, I'm also using it to write data to a sheet in an Excel workbook. I write the necessary data out to the Data
sheet:
First hackintosh with Windows dual boot using Intel NUC DC3217BY
Mac OS X
My friend recently introduced me to the Intel NUC (DC3217BY). It's basically a micro form factor barebone system that comes with Intel's ULV i3 processor (powerful and low power consumpton). I decided to get one, slapped on 8 GB of ram, a 256 GB mSATA SSD, and a Broadcom based half-sized e-PCI network card, and Hackintosh'd it since the processor is similar to what's in an Apple Macbook Air. Basic instructions for this particular hardware could be found here and here. A generic guide could be found here. For dual booting with Windows, this article and this post helped. This is what I recalled doing to set up:
Bulk resize images and keeping original files
Linux
Suppose you have a directory (with subdirectories) full of images, and you want to resize them all while keeping the original images. To do so, first create a copy of the directory tree without the image files. Then, using a for loop, find each image file and apply the
convert
command to it. The following is an example to resize jpg
files to 40% of the original quality.
Amp and USB Chargers
Android
Hardware
This is a good article that explains how USB charging works. Basically, avoid cheap chargers. For any reasonably good charger, the amperage of the charger doesn't really matter so long as it exceeds what the device requires; that is, use a charger with at least 0.5 Amp if the device requires 0.5 Amp (what the original charger uses). Thus, it's OK to use my 2 Amp chargers on most of my mobile device so things charge faster!
Skeleton to create fast automatic tree diagrams using R and Graphviz
R
Statistics
I've had to create tree diagrams (dendrograms, decision trees) many times in the past to illustrate the flow of data or decisions (e.g., data flow for a study). This is usually a manual task done in MS Powerpoint or Visio. I've also made some diagrams in the past using Graphviz based on the DOT language to make creation more reproducible. However, that still felt pretty manual.
Hosting multiple WordPress sites with WordPress Multisite and Domain Mapping
Internet
I use WordPress to run my blog. I recently had the need to run another site and wanted to also use WordPress as my CMS. However, I don't want to run another installation of WordPress if I don't have to. I followed the directions on here to create enable WordPress Multisite, and followed the directions on here to make use of a different domain name for my second site. Basically,
Extend wifi network with a DD-WRT router as a wireless access point
Networking
My house is two stories so my router's wifi signal can be spotty upstairs or in the backyard. Since I run network cables to each of my room, I'm able to extend my wifi signal by placing a DD-WRT router upstairs and near the backyard by configuring them as Wireless Access Points.
Delimited file where delimiter clashes with data values
Coding
Python
R
SAS
Linux
A comma-separated values (CSV) file is a typical way to store tabular/rectangular data. If a data cell contain a comma, then the cell with the commas is typically wrapped with quotes. However, what if a data cell contains a comma and a quotation mark? To avoid such scenarios, it is typically wise to use a delimiter that has a low chance of showing up in your data, such as the pipe ("|") or caret ("^") character. However, there are cases when the data is a long string with all sorts of data characters, including the pipe and caret characters. What then should the delimiter be in order to avoid a delimiter collision? As the Wikipedia article suggests, using special ASCII characters such as the unit/field separator (hex: 1F) could help as they probably won't be in your data (no keyboard key that corresponds to it!).
Automatically capitalize or uppercase or expand keywords in Emacs using Abbrev Mode
Emacs
I like that SQL Mode in Emacs comes with an interactive mode that I could execute a query in a buffer to a client buffer similar to how I could execute R code using ESS. However, I don't think SQL mode is that great at formatting SQL code (eg, indenting). I guess I could live with manual indenting and selecting in multiple lines (preceded by a comma).
Find text or string in files of a certain type
Linux
Mac OS X
One can use
grep "mystring" myfile.ext
to find the lines in myfile.ext
containing mystring
. One could also use grep "mystring" *.ext
to find mystring
in all files with extension ext
. Similarly, one could use grep "mystring" /directory
to search for mystring
in all files in the directory. What if one wants to search for mystring
in all *.ext
files in a certain path /directory
? Most posts online would suggest something along the line of
Avoid data truncation in SAS when importing CSV files
SAS
SAS's
Proc Import
is great for importing a CSV or other delimited files:things just "work" most of the time. We don't need to specify variable names, variable type, etc. However, data truncation or mis-matched variable type can happen as the procedure determines the data type and length of the variables based on the first few rows of the delimited file.
SAS Proc SQL Group By returns multiple rows per group
SAS
SQL
Coding
Just wanted to note that for traditional SQL implementations (e.g., MySQL, MS-SQL), the
Group By
statement used to aggregate a variable by certain variable(s) returns 1 row for each group. When a column that is not unique within a group is also selected, then the row that's returned is determined somehow by the DB software.
Fixing group permissions in sshfs via OSXfuse
Mac OS X
Linux
One could mount SSHFS partitions on a Mac OS X system via OSXfuse and MacFusion to access remote files securely as if the files were stored locally. However, group permission seems to not be preserved (get a
permission denied
message). This post shows how one could fix it by using the -o defer_permissions
options.
Screen brightness after suspend in Ubuntu
Linux
Many laptops have their screens dimmed after returning from "suspend" and cannot get back to their original brightness. The bug hasn't been fixed for 3 years. A fix is provided in the bug report by putting something like the following in
/etc/rc.local
:
Truly live Linux distribution on USB with Casper
Linux
Many distros come with the ability to try or use the distro via a live session on USB or CD without installing it to disk. However, the live session starts fresh with every boot. I recently discovered Casper. It allows you to have a live session on USB with all work saved on USB.
Better decision tree graphics for rpart via party and partykit
R
Statistics
I've been using Graphviz to create better decision tree graphics "by hand" for
rpart
objects created in R
(final tree). I stumbled on this post that shows how one could convert an rpart
object to a party
project via the as.party
function in partykit
to utilize the plot functions in party
. It looks quite nice.
Enable root account in Ubuntu?
Linux
After my recent experience with broken
su
and sudo
commands in a failed system upgrade, I realized that although disabling the root account has many advantages, one of the disadvantage is that I can't login as root in the terminal when I'm physically in front of the system. This is a major issue if su
, sudo
, and passwd
binaries are broken somehow. Luckily, chroot was there to the rescue for me. Now, I contemplate whether I should enable the root account on my systems…
Flipping the classroom: creating screencast lectures in Linux
Teaching
Internet
Linux
Statistics
I'm debating the idea (hype) of flipping the classroom for one of my classes next Fall where students watch lecture videos at home (or elsewhere) so I could spend class time doing more hands-on activities like discussing the art of data analysis and how to solve problems with statistics. I think Khan Academy, Udacity, and Coursera are doing a great service for humanity by offering high quality courses taught by excellent teachers online that are accessible to anyone with an internet connection.
Use wget or curl to download files that require login
Linux
Internet
Some sites require a user to login or authenticate before a file could be downloaded. To do so with wget, make use of the
--post-data
and --save-cookies
arguments as described here. We first have to view the source of the html file, determine the name of the variables to post, and pass these information onto the site via the wget
. Once we have a cookie file that's authenticated, we can use this cookie file via the --load-cookies
argument to download our files.
Build 32 bit R on 64 bit Ubuntu by utilizing chroot
R
Linux
In the past, I've described how one could build multiarch (64 bit and 32 bit) versions of R on a 64 bit Ubuntu machine. The method based on this thread no longer works as of R 2.13 or 2.14 I believe. I received advice from someone on
#R
over on freenode
(forgot who) a few months ago that suggested the chroot route (see this also). I recently tried it and wanted to document the procedures. Although the solution isn't as nice as the previous multiarch route, it will suffice for now. With the chroot
method, first compile the 64 bit version of R the usual way. For the 32 bit version of R, do:
Test ram with Memtest86+ and ignore bad parts with badram in grub
Linux
Recently, my computer kept freezing whenever I started conkeror (with 100+ buffers loading from a previous session). Folks over at
#conkeror
on freenode
suggested that the problem might be due to faulty ram. They suggested testing my ram with Memtest86+. It is installed by default on Ubuntu.
Beware: certain Amazon EC2 AMI’s terminate at shutdown
Linux
Internet
I toyed with Amazon EC2 last year by migrating my web server there. After issuing
shutdown -P
in Ubuntu last week, my instance disappeared from the list in the AWS dashboard. I seeked help and found out that some AMI's are set to terminate at shutdown. Unfortunately, the AMI I used was one of them. Files and settings are not recoverable after an instance is terminated. Luckily, I transferred most of my critical information out prior to the shutdown; I'm only missing one file. Note to self: for all future instances, the shutdown behavior to stop
(as opposed to terminate
) could be modified per these instructions. First, create and download the X.509 certificate and private key. To access the EC2 api, do the following on your computer:
Startup applications in LXDE: caps lock as control
Linux
I just installed Linux Mint on my Asus T101MT. As with all my computers, I prefer to have the caps lock key as a control key. Of these methods, only the
xkb
command works. To have the key remapped every session, I followed these instructions and created the file ~/.config/autostart/nocaps.desktop
:
Protect root on Android devices with stock roms
Android
I just discovered Voodoo OTA Rootkeeper, which makes a backup copy of root (
su
) and could restore root after any OTA update. I guess this is useful for me since I haven't installed any custom roms on these two devices (yet) and am able to do OTA updates. The app does not work with all devices, but it works on my HTC Evo 3D phone and Asus Transformer Prime tablet.
Learn regular expression by doing with regetron
Coding
When it comes to learning, I buy into the "learning by doing" philosophy. This is especially true when learning a computer tool or programming language. I stumbled upon this regex tutorial that makes use of a set of exercises in conjunction with regetron. Time to hone in on my regex skills.
Break CAPTCHA with cheap labor via services like antigate
Internet
I read this interesting post the other day about how to break those annoying CAPTCHA's by utilizing cheap labor (like $1 for 1,000 captchas) from less developed countries with services like Antigate. Kind of a cool thought, but that would just mean it's easier for the spammers to work.
WPS flaw on routers allows WPA protected WIFI networks to be cracked
Linux
Networking
This post discusses how one could use Reaver to make use of a flaw in WPS to recover a WPA password. Tomato and DD-WRT firmwares don't support WPS so my network is safe. Lesson: buy and use a router that you can flash Tomato or DD-WRT.
Annotate or write on top of a pdf file
Linux
Sometimes I need to annotate a pdf file, either to take notes or to fill it out as a form since I have ugly handwriting; for the latter, I'm referring to the case where the pdf file does not have form fields you can type in with Adobe Reader. This post describes some programs for annotating pdf files. Xournal is useful when you just want to write on top of a file and print out the content; it converts the content of the original pdf file to an image and saves the old and new contents as a pdf file. I use Xournal on my Asus T101MT touchscreen netbook. Okular seems good except that you can't save the annotated content and original content as one file or even print the annotated content with the original content.
WiFi Tether on HTC EVO 3D
Android
Owned my HTC EVO 3D (rooted, stock rom) for almost half a year already and I have not gotten WiFi Tether working until now based on this post. I'm currently using version 3.9 Beta 9. To get tethering to work, I need to enable MSS clamping and Routing fix under Menu > Settings per this post. Yay! I can now tether when I'm away from home.
Access or stream remote files on Android using EStrong File Explorer
Android
I currently use AndFTP to download files remotely using the ftp or sftp protocol. For local file access, I use something like ASTRO File Manager or Linda File Manager. I recently discovered EStrong File Explorer, a file manager that lets me access both local and remote files. You can access or stream remote files directly without downloading it first. That is, you can select an mp3 file and play it with your favorite music app directly. It is like the remote server is mounted on the device like in Linux which makes file access feel seamless. Too bad streaming movie files like XviD is a little laggy with MX Video Player or Rockplayer Lite even when the server is on the local network.
Play music in background or while multi-tasking on Android with doubleTwist
Android
When I stream podcast files from Google Reader or play mp3 files, I like to be able to work on emails and other tasks. Most default music apps stops the music when another app is accessed. I found doubleTwist after some searching and it allows me to play music in the background. I don't use it for it's advertised features, syncing with iTunes.
Public data sets
Statistics
Researching
Teaching
The Washington Post keeps a list here. There is also the UCI Machine Learning Repository.
Using Google Adwords to max out Dropbox referrals
Internet
I stumpled upon Vladik Rikhter's post via Hacker News on how to utilize Google Adwords to max out your Dropbox referrals for less than $10. Basically, you set up an ad campaign that leads people to sign up for a Dropbox account via your referral link, giving them 2.25 GB (250 MB extra for using a referral link) of free storage while you benefit 250 MB (or 500 MB if your account is linked to your school email). My initial attempt at this did not yield any clicks, probably because many people were following this method. I waited two weeks and I still didn't get any referrals; this was probably brought forth by the publicity generated by Lifehacker's post (see this too). After about 3 months of waiting, I raised my max bid to $0.10/click, and I got 5 referrals in a 24 hour period. I then raised my max bid to $0.20/click and I completed my quota in a day (I'm at 18.3 GB right now with my student email linked to my account). My total cost was $5.34 since I had many referrals prior to using the campaign (was at 9.3 GB). FYI, the default auto bid was around $0.14/click, so this amount should give you enough referrals in a 24 hour period.
Emacs 24 crashing when launched with Synapse and an emacsclient window is closed
Emacs
Linux
I'm currently using Emacs 24 (pre-release) installed from this repository on Ubuntu 11.04. An error (crash: segmentation fault) that arose after moving from Emacs 23 to Emacs 24 is as follow (could stem from the pre-release repository; might not be the case if I compile from source).
Mount box.net on Ubuntu Linux via webDAV
Linux
Internet
I recently signed up for a free 50gb account at Box.net. At the time of this writing, Box.net does not have an application to sync files from a Linux machine to their servers. I followed this guide and this post to mount the service to a local directory. I did:
Update Wordpress plugins via sftp (ssh)
Internet
I have an FTP server running on my web server to easily update plugins via the web admin page since the default Wordpress install only allows updates to be made via FTP and FTPS. FTP is an old protocol that is insecure. To update via sftp (ssh), install
libssh2-php
in Ubuntu and restart apache (credit here).
Filtered response in gmail
Internet
I use my gmail account to manage multiple email accounts/addresses. I recently wanted to set up a vacation responder for one of my email address. However, the default vacation responder in gmail uses the default email account to send and there is no option to activate vacation responder for a single email address. I found this post on lifehacker that shows how one could set up a flexible auto response system with the use of a gmail account dedicated to auto response. Let gmail 1 be my primary account, and gmail 2 be my dedicated account. Basically,
PBX in a Flash + Incredible PBX makes setting up FreePBX + Asterisk easy
VoIP
Asterisk is a very powerful open source telephony platform. I've described how to install it and configure Google Voice with it in the past. Setup is quite complicated for a newbie to get started. I was never able to set it up to the point for me to connect to the Asterisk server using the SIP protocol to make calls. Luckily, there is FreePBX which makes configuring Asterisk a lot easier. Even better, there are PBX in a Flash (PIAF) and Incredible PBX (choose different PIAF versions) ISO's that make creating a PBX sandbox server quite easily. With these tools, I will be able to have my own conference call number and fax number for personal use.
Skip disk mount errors in Linux
Linux
I automatically mount disk drives using
/etc/fstab
. However, when I boot a system with a disk removed, the system will give a mount error, giving me the option to skip (S) or manually (M) mount the disk. This doesn't work well for headless servers. To skip the mount errors during boot, this suggests using the nobootwait
option in /etc/fstab
. Examples can be found here. For me, I have something like the following:
Automatic email notifications for Google Docs Forms
Internet
I use Google Docs to collect data using the Forms feature. People can just fill out the form and data will be automatically entered into a spreadsheet for me. It really makes data collection easy in simple cases. For example, creating a contact form using Google Docs is fairly straightforward. Google can even notify you when someone submits a form. However, the notification feature is weak in that it doesn't provide any useful or customizable data; it just lets you know someone submitted a form. Luckily, Google has a scripts feature. This mail merge feature shows how you can email a list of people individualized emails. To get more useful notifications, you can write a script that will run whenever submits a script. This is a very useful example. I'll reproduce it here:
Create a website instead of a blog from Wordpress using the right theme
Internet
Wordpress is no doubt a great blogging platform. However, it is also a user-friendly Content Manangement System (CMS) that can be used to run a website. However, you have to tweak it to not look like a blog and more like a traditional website. The easiest way to do so is probably through an appropriate theme. I ran into the Phanatics theme, which looks very asthetically appealing with a monetary fee for its license. I also found the free Weaver theme to also be good looking.
Anonymous web-surfing with Tor
Internet
I wrote about Tor in the past but I never tried using it. I installed it on my Ubuntu laptop and tested using it via Firefox and [Torbutton]. Now that it's set up, when I want to surf the web anonymously, I just need to launch Firefox and toggle Torbutton to be on. Even though the Tor daemon is running in the background, I won't use it on a day to day basis.
Install and update multiple Linux distros using NetbootCD
Linux
This post on lifehacker introduced me to NetbootCD, a live CD that allows me to install the latest version of popular Linux Distros (e.g, Ubuntu, Debian, CentOS, etc.) using a single live cd or usb drive. The latest version of the chosen OS is installed over the network. This will save me lots of time and space from repeatedly downloading the latest OS version.
Lightweight Portable Security Linux Distribution by the US Department of Defense
Linux
While on the topic of security (been blogging about encryption and such pretty often in the last month), I remember reading about a basic, secure linux distribution for web browsing created by the US Department of Defense. I downloaded Lightweight Portable Security just in case I need to use a secure OS on someone else's hardware.
Backup, re-install Ubuntu with full disk encryption, and restore all files and settings
Linux
Security
When doing serious work like surfing the internet, writing, or programming, I like to do so from a single user interface regardless of whether I'm at work or home. Currently, this takes the form of a Linux laptop (Ubuntu) due to portability (laptop), power (Linux/Ubuntu), and the availability of a keyboard and touchpad (fast input).
Remote unlocking LUKS encrypted LVM using Dropbear SSH in Ubuntu
Linux
Security
I recently performed a full disk encryption on my server using dm-crypt + LUKS. I did not address remote unlocking of the disk then because I did not know how. Remote unlocking is highly desirable I might not be physically near the server when a restart is necessary.
S5 Reloaded themes with Sigma’s emacs org-mode method
Emacs
Internet
Linux
In the past, I've described the benefits of a slide show based on html. I used to prefer the html5 method over S5, but after some use, I've come to realize that the html5 method (at least the one described here) isn't ready for prime time yet. The display of the slides can get screwy. For example, the current slide might not be centered correctly. Therefore, I'm going to recommend the S5 system for html slide shows for now.
MKV on PS3 using Showtime
Hardware
Networking
I've described how one could play MKV files on the PS3 before. Another method is to play them using Showtime, a media player that can be installed on a jailbroken PS3; it supports MKV natively. This method is better than my previous solution because on-the-fly transcoding is not necessary and hence, the fast forward and rewind capabilities of the video are not lost.
Strong (long) passwords
Internet
Linux
Networking
Security
REMINDER: PASSHPHRASE SHOULD BE 24+ (42+) CHARACTERS LONG TO BE EQUIVALENT TO A 128 (256) BIT KEY.
Offline maps for GPS on Android without a need for a data connection
Android
To follow up on my recent post about Navit, a cross-platform map and GPS program that you can use on your laptop when on the road, I did a search for map/GPS programs on the Android platform that do not rely on a data connection. Like I mentioned in my previous post, I sometimes travel to locations where I do not have cellular signal (like in the mountains), which render Google Maps and my phone's default navigation software useless. GPS signals are usually available as long as I'm outdoors (based on my personal experiences), so I hope to rely on a device with a GPS receiver with maps stored on it.
OpenDNS on Tomato router for faster web experience
Networking
I saw this post and decided to opt to use OpenDNS as my DNS server instead of my ISP's server. I followed these instructions to set it up. Basically, I activated a second DDNS service (Basic > DDNS) and entered my login information for OpenDNS. When activating it, I set it to update my dynamic IP and to use it as my DNS server. It was as simple as that.
Home address on GPS navigation device
Hardware
Note to self: do NOT enter your actual home address on your GPS device. If someone how gets a hold of the GPS device and your keys, they can rob you pretty easily. Rather than enter your home address, enter the address of a nearby business. That way, when you want to travel home from afar, the GPS device will tell you how to get back to the vicinity of your home.
Navit: a map and navigation program
Linux
Although I own a GPS navigation device and have GPS capabilities and a data plan on my cell phone, I prefer to have a mapping program on my laptop just to be safe as I've been to many places where I don't receive cellular signal. Back in 2004, MS Mappoint was installed on my Windows XP laptop. I liked it because the entire USA map was accessible on my laptop without an internet connection. In addition, I could enter the start and end destinations and it would give me driving directions. I asked for Linux alternatives at the LQ forums and people recommended tangoGPS. More information about tangoGPS can be found here. It seems like a solid program as you can plug in a GPS receiver and use your laptop as a GPS device. The downside is that maps are not stored on the computer so an internet connection is required. However, you can cache maps of the area you will be heading to ahead of time. This is inconvenient and remembering to do so is unrealistic for me.
Build multiarch R (32 bit and 64 bit) on Debian/Ubuntu
Linux
R
I have the 64 bit version of R compiled from source on my Ubuntu laptop. I recently had a need for R based on 32 bit since a package I needed to compile and use only works in 32 bit. I thought it was readily available on Ubuntu since both 32 bit and 64 bit versions of R are shipped with the Windows and Mac OS X installers. I tried figuring out how to do so using the manual (R 2.13.1), but could not figure it out on my own. I seeked help on R-devel and received some helpful responses.
sideway or landscape table over multiple pages in LaTeX
LaTeX
Most of my results table are huge. To get them situated in a document, the
sidewaystable
environment from the rotating is helpful in getting a wide table to fit on one page. To get a long table to span multiple pages (regular orientation), one can utilize the longtable environment; see this example.
Ubuntu USB installation - detect and mount cd-rom failed
Linux
I recently had to reinstall Ubuntu 10.04 Server 64 bit on one of my computers. I did it before, but this time, I could not proceed with the installation at the "detect and mount cd-rom" step. Because this step didn't complete, I couldn't continue the installation (I guess files aren't accessible). A solution is described here. However, it didn't work for me (couldn't mount the usb drive for some reason). I believe the error pertains to Ubuntu 10.04 and 10.10. I downloaded 11.04 and the this error was not there. After the installation screen booted up I noticed my keyboard failed to work. I had to plug it into another usb plug for it to work. I think I figured out why this "cd-rom" issue came about in 10.04 in the first place (I successfully installed it before). Before, I used a usb 2.0 port. This time, I used a usb 3.0 port. I think once the Linux kernel booted up, it failed to see the usb drive anymore. Hence, it was looking for at the cd-rom drive. Note to self: always try multiple usb ports!
VPN service for an anonymous or untraceable internet presence
Internet
Networking
Security
I recently considered the use a paid VPN service to connect myself to the internet. Why? The internet is a whole other world out there, and you just don't know how much privacy you lose with all the connections you make on your computer. I see myself and the average user at a disadvantage when it comes to privacy because we aren't savvy enough to know the underlying workings of the internet. The transmission of data packets from one device to another gives rise to the opportunity for a knowledgeable person (not me, of course ;) to decipher private information in that transmission. I'm not even referring to people stealing my password. It's just scary to know people can find out what sites you've visited, what services you use, etc. Your privacy can be compromised without you even knowing it. People can figure out your daily habits and make judgment on the kind of person you are. This is quite scary. For example, your internet service provider (ISP) knows exactly what files (unencrypted) you are transporting on the internet: the source, the destination, the timing, duration, frequency, etc. However, if your connection between two devices are encrypted, for example, using SSL, then all the ISP (and other snoopers) see is a stream of data that have no idea what it is without the proper key.
Working with SAS macro variables
Statistics
Believe it or not, I recently had to use SAS for a simulation study because it was the only system available that could maximize the exact partial likelihood of a Cox proportional hazards model for tied data (
ties=exact
) in a reasonable amount of time for a data set of 2,000 observations. This was the first time I ran a simulation in SAS. I made use of the MACRO capabilities of SAS to write generic functions that could be used to repeat steps; I might blog about my overall experience some day. Today, I'll just write about one source of frustration that I had.
research paper management or library with emacs
Emacs
Researching
Despite the availability of softwares such as Mendeley, Zotero, and JabRef, I like to store my papers (
pdf
files) and citation information (bib
files) using a directory stucture, enter notes into a text file (org-mode
), view notes and bibliographic information using a single file, cite references in LaTeX from a single bib
file, and manage papers in Emacs using dired
. I choose to manage my references using the bib format because it is the de facto standard in academia (at least in my field), and hence, is easily exported from the publisher's website. In addition, I write all my papers using LaTeX for which I utilize bibtex
whenever a reference is made; if I were to write papers using another program that utilizes a different format for bibliographies, I can easily convert the bib
files using bibutils.
archive and synchronizations with unison and rsync
Linux
I use rsync as my primary backup utility as it is fairly efficient at archiving files (NOTE: to backup to fat32 usb drives, make sure
--modify-window=1
is used). I recently discovered unison, and am using it for real time synchronization between directories from two Linux computers. I really like it.
Web-based collaboration of LaTeX documents
Internet
LaTeX
This post brought LaTeX Lab to my attention (again). It is a service based on the Google Docs platform for writing and collaborating LaTeX documents. I also became aware of ScribTeX, another similar service. I'm not sure how often I'll use these services as I'm more (counter?) productive in emacs. Moreover, my interdisciplinary collaborators don't use LaTeX. As for the collaborators that do use LaTeX, we just email the
tex
file back and forth as usually only one person edits the file at a time. This post describes how I view changes to a tex
file.
Real time file synchronization like Dropbox via Unison
Linux
Security
Dropbox is a very nice tool for real time synchronization. It works very well to keep files from multiple devices (computers, phones, etc.) in sync. I use it mainly as a cloud-based backup for some of my files. However, it's been on the headlines recently due to security and privacy concerns, leading to calls for encrypting your files prior to syncing with Dropbox.
web-based ssh
Internet
SSH has come to be my most used protocol besides http. It allows me to fully access a computer, transfer files, and all communications are encrypted (so I feel safe!). Since all my computers are Linux-based, I can use ssh by starting a terminal. However, what If I'm on a computer without an ssh client, say, a Windows machine? Sure, I can install putty on a Windows computer when I need to ssh. Is there a more convenient way?
Adobe Flash Player 11 beta out - no more npviewer.bin process sucking CPU resource?
Internet
Linux
Adobe Flash Flash Player 11 beta is out. I immediately installed it per these instructions after downloading the binaries here (via the
~/.mozilla/plugins
method). I am so sick of the npviewer.bin
process always sucking up my CPU resource. So far, I don't notice this issue anymore. I really dislike the fact that Adobe doesn't bother to fix bugs in their products for Linux users. Because of this, I hope all websites implement utilize HTML5 so I don't have to depend on Adobe!
My own programming style convention for most languages
Coding
R
I write code mainly in R, and from times to times, in C, C++, SAS, bash, python, and perl. There are style guides out there that help make your code more consistent and readable to yourself and others. Here is a style guide for C++, and here is Google's style guide for R and here is Hadley Wickam's guide for R. For R, I agree more with Google's style guide than Hadley Wickam's because I absolutely hate typing the underscore (personal preference) and because Google's style guide seems more related to that of the C++'s guide. Style guides differ by languages because the languages are different (restrictions on names, etc.).
Ignore errors when executing a command in emacs with condition-case
Emacs
Suppose you want to execute a command in your emacs init file, but this command sometimes return an error. For example, you ask emacs to open a file for you, but the file doesn't exist. When an error occurs, the rest of the init will not be loaded. Once can make use of the
condition-case
command to ignore the error. The following is an example:
Wordpress’s Syntax Highlighter Evolved messes up my code
Internet
I use email to post entries to my blog. I place code snippets in between "LEFT BRACKET plain RIGHT BRACKET" and "LEFT BRACKET /plain RIGHT BRACKET" with the "div" tag surrounding them. My code gets displayed fine using the Syntax Highlighter Evolved plugin. However, when I edit the entry in wordpress, quotes, ampersands, less than signs, and greater than signs get converted to html code when I save the post and it makes my code unreadable. This also happens if I start writing my post in wordpress. This only happens to text within the code chunk, and not text outside of it. This conversion that happens when saving does not happen when the plugin is disabled. I did not find a fix for this.
Determining number of nodes or cores available in an SGE Queue
Linux
To determine the status of a queue in SGE, one can issue the command
qstat -g c
to get such information like number of CPU available and the current CPU and memory load. However, this information can be misleading when nodes can be cross-listed in multiple Q's. A Q can say X number of nodes are unused, when in reality, they are in use in a different Q. Consequently, a submitted parallel job asking for X cores can wait in limbo for quite some time depending on the cluster's load. The following sgeQload.R
R script uses some commands explained in the cheat sheet to output the number of cores really available:
Modify spacings in listing (itemize/enumerate) environments for LaTeX Beamer
LaTeX
I don't usually worry about display options in LaTeX since it does a good job most of the time. However, when I write presentations using LaTeX Beamer, I make use of the
itemize
and enumerate
environments quite often to list ideas, and the default setting in most of the templates lack generous spacing between items and between nested lists. The enumitem is the preferred method to modify spacing properties in list environments for LaTeX documents. However, Beamer has its own definitions for these environments for use with overlays, etc.
Google Voice on a telephone without a server
Hardware
VoIP
I already discussed how one can make use of Google Voice with Asterisk – the possibilities are limitless. However, all this requires a server running Asterisk. I recently explored how one can explore other options of Google Voice (or other VoIP services) without the use of a server.
Version control and collaborating with LaTeX files
LaTeX
This post finally pushed me to explore ways to version control and collaborate with others using LaTeX files. I'm assuming the collaborators also use LaTeX, which is rare in itself when your primary collaborators are scientists that work mainly with WYSISWG editors, in particular, MS Word. I will outline how I collaborate with non-LaTeX users in a future post.
Byobu, an enhanced screen
Linux
I use screen on a daily basis. I recently discovered byobu, an enhanced add-on for screen that makes it even more customizable. A screencast can be found here. After reading some descriptions of
byobu
here and here, I re-discovered screen's ability to split screens. I'll probably begin to use this feature. Not sure if I'll need to use byobu
. I do like the row on the bottom describing the system resources.
Non-LaTeX presentations using org-mode - S5 and HTML5 slides
Emacs
I recently had frustrations with presentations written using MS Powerpoint; I'm not even going to mention KeyNote as it is only available on a Mac. LibreOffice is my WYSIWYG editor on my Linux machine for writing quick and nicely formatted content. I also have MS Office installed on my machine using WINE. I sometimes use Google Docs for presentation when I have to collaborate. Most of the times, viewing MS-generated files on LibreOffice suffices. When it doesn't work well, I opt for MS Office. However, I recently had to open a powerpoint presentation with an audio link. It didn't play on LibreOffice and it didn't play on my WINE MS Office. I started thinking about a possible route for presentation files that are self-contained and cross-platform. PDF comes to mind first, but to do so would mean I have to use Beamer to generate the slides or to use Adobe Acrobat to create the slides. I don't want to do either as I'm referring to the scenario of creating quick slides.
Watch a folder or file with inotify-tools
Linux
I recently needed to watch a folder for a file. I stumbled on this post and discovered inotify-tools. Using it, I can write a bash script to watch a folder for files, and do things when certain events happen. The "watch" script can be launched at startup by adding its execution in
/etc/rc.local
.
SSH-encrypted proxy for Android or when connected to a public wifi
Android
Linux
Security
This post describes how one can set up an ssh-encrypted proxy for browsing the web when connected to a public wifi. This post describes how to do so with an Android device. The gist is to have your proxy server request all internet contents, and pass over the information to you over an encrypted ssh tunnel.
Add a printer on the command line - cups-pdf example
Linux
Suppose you need to add a printer to a computer on the command line, e.g., a server that you access via a terminal. This post outlines how to use some basic commands from the CUPS system. I will illustrate with commands for a network-connected printer and cups-pdf for printing to PDF files (
~/PDF/
):
Google CloudPrint on Linux
Linux
Google CloudPrint allows users to print to the cloud from devices such as your mobile phone. That is, you can print documents and pictures from your phone to printers you have access to (e.g., at home) but is not directly connected to your phone. As of 5/12/2011, only printing from mobile Gmail and Google Docs (from the browser) are supported besides the Chrome OS. Also, only printers that are accessible to computers running Windows or Mac with the Chrome browser installed are supported. What about Linux? This thread points to the cloudprint python program. I installed it and have it started at startup via the following:
sshfs doesn’t forward X + conflict with “ControlMaster auto”
Linux
I use sshfs to mount remote directories on my laptop to access files as if they are stored locally. I love it. I also use the ControlMaster feature of OpenSSH to reuse an existing ssh connection when opening new ssh connections to the same host.
emacs keybindings in xpdf and xdvi
Emacs
I've been using xpdf and xdvi for reading documents I've downloaded or my LaTeX-generated documents on the laptop more and more these days due to their speed. The one thing I require (desire) in all programs I use on a day to day basis, especially when I have to navigate the file, is to have emacs keybindings.
Writing and publishing books on Kindle
Linux
Thanks to the digital book revolution, it is quite easy to publish your work yourself without convincing a publisher to publish them for you or pay hefty fees to have them publish. This post on Linux Journal suggests using an OpenOffice template (not free) to write your work that is perfectly compatible with the Kindle.
LPR printing on command line
Linux
These days, with Linux distros such as Ubuntu, printing is quite easy. You can add and remove printers and print files just as easily (or even more so) compared to Windows or Mac. Sometimes I might just want to print from the command line. To do so, I make use of the
lpr
command.
Testing a hypothesis and hypothesis generation
Statistics
Teaching
I like this podcast. It discusses some recent findings about genes possibly relating to Alzheimer's Disease. In it, the guest speaks of not having a hypothesis going into the study. Then the host (or another guest) raised the question of conducting a study without a pre-defined hypothesis. The keyword was "hypothesis generation." I'm happy this was brought up on NPR (although this wasn't the point of the show). I might use it as an example in class some day.
Amazon EC2 free hosting (micro instance) for 1 year - Ubuntu + Wordpress + Asterisk
Internet
Linux
VoIP
I use old laptops as web servers at school. However, they really aren't built to be running continuously; I have disk problems on both of my old laptops. Amazon Web Services offer a free usage tier of EC2 cloud service for 1 year. After the 1 year, the micro instance cost about $.02/hr, which is still pretty cheap. I sign up for an account with Amazon Elastic Compute Cloud (EC2), started an instance, installed the ami-3e02f257 AMI, downloaded the private key, and the ball started to roll. Before you can ssh to your server, you have to set up the firewall from the management console of EC2 to open port 22 per this post. To log in and set up, I had to download the private key (PEM) that amazon generated and logged in as the user
ubuntu
(root is not allowed for my chosen AMI; this is the default user per the AMI):
Xterm - Alt key and copy + paste
Linux
I recently went back to using xterm as my default terminal since in Gnome-Terminal, Alt-V gets sent to the menu, and not to emacs when it is running. In Xterm, the Alt key did not work immediately; it gave me wierd characters. This post shows me how to fix the problem (put in
~/.Xresources
instead). Also, copy and paste does not work intuitively by default. This post shows a fix. Both involves tweaking the options in ~/.Xresources
.
Latest Asterisk on Ubuntu
Linux
VoIP
I recently changed my web server, and had to re-install Asterisk. I wanted to compile from source but I had an "input/output" error, which means either my drive is bad or I'm having issues with the kernel. SMART tells me my drive is OK, so I assume it's the kernel.
Foxit Phantom, an Adobe Acrobat alternative, for Linux via WINE
Linux
There are plenty of open source tools on Linux that can assist with pdf files. However, sometimes you just have to do something that requires Adobe Acrobat (e.g., editing pdf files). You can't install Adobe Acrobat directly on Linux; I currently use Adobe Acrobat via a Windows virtual machine via VirtualBox. This post lists some popular pdf tools, and among them are tools from Foxit. The reader can be installed on Linux, but not Phantom, their Acrobat alternative. However, you can install the Windows version through WINE.
escreen instead of elscreen for screen-like features in emacs
Emacs
Linux
I've been using elscreen for screen-like features in emacs the last couple of years. However, I have a few complaints. Elscreen has issues when used with
emacsclient -c
: a new frame might not be created from the -c
argument, which messes up my current screen in emacs, and the "Opening Server Files Always in a New Frame" tip on this did not resolve this issue perfectly with elscreen. Also, when used with emacs' Desktop mode, I get an error when launching emacs with an Rnw
file (Rweave, or Sweave) is in the file list; the launch actually does not load all files, and I have to exit and delete my desktop file (which I'll lose the list of files).
File associations in Linux (Gnome and Mailcap)
Linux
File associations in Linux can be tricky. On my Gnome-based Ubuntu machine, I can associate a filetype by right-clicking a file in Nautilus, and slecting "Open with". To modify the defaults using config files, this post describes it well. Basically, the two main locations to check out are
/usr/share/applications/defaults.list
and ~/.local/share/applications/
. File associations made using these methods apply to the commands gnome-open
and xdg-open
as well.
Block/disable transition ads (Ad-brite)
Internet
Linux
Some sites I go to forces a full page ad once I'm visit a page for a few seconds. After a few additional seconds, I can click on "skip ad". I find this very annoying as the ad loads whenever I go to a new page within the website. I recently found out that these ads were called "full page ads" or "transition ads", ran by Ad-Brite. After some googling, I found this solution, which works. On the
hosts
file (on a Debian-based machine, /etc/hosts
), add the following and things should be fixed immediately (restart on a Windows machine):
Dissociate C-i, C-m, and C-[ in emacs
Emacs
Linux
Every now and then, I bind functions to keys such as
C-m
, C-i
, and C-[
, but run into issues as these keys are identical to RETURN
, TAB
, and ESC
. See this and the named ascii list for more information. I tried many solutions before but never got them to work. I finally ran across a solution that works, namely Caio's solution on this post. At the time of this writing, it shows as the last place answer on stacked overflow, so I (and many others) must have missed it. Note that this works only on GUI instances of Emacs and not terminal ones. I will reproduce the solution:
Burn CD’s and DVD’s from emacs using dired and dvd+rw-tools
Emacs
Linux
I recently explored burning discs from the command line. To make things even more convenient, I can employ emacs dired to help in selecting the files. That is, suppose I have some files in a directory opened in dired. I can mark the files I want to be burnt, and invoke
!
(dired-do-shell-command
) on those files with the command:
Burn dvd from the command line
Linux
I recently had to back files from my Ubuntu-based NAS. I initially used xfburn, a medium-weight burning software. It does a pretty good job. However, I decided to burn via the command line to refrain from pointing and clicking. I found dvd+rw-tools. Some simple examples are found here; the full filename solution was found here. Suppose the dvd burner is
/dev/sr0
(found from scanning through dmesg | less
). Then I can burn files with:
Emacs as my default pdf viewer
Emacs
Linux
In the last week, I've switched over to using emacs for pdf viewing (reading) via doc-view because I noticed "C-v" and "M-v" did not do what I want in Document Viewer aka Evince ;). At least with OpenOffice, I could remap the navigation keys; not so with Evince. I usually open the pdf files in emacs via dired, but I'm loving it so much that I want it to be the default viewer on my system, i.e., double clicking files in Nautilus. I created the following bash script and set it as the default pdf viewer on my Ubuntu machine.
Convert raster image to vector image
Linux
I wanted to convert some raster images (bmp, png, etc.) to vector images (svg, pdf, eps) so that they can be rescaled easily. I found Vector Magic, which used to be free from Stanford University but now cost a grip; however, they do allow some free conversions. This post let me know that Potrace is implemented in Inkscape. I just used Potrace directly since it is command line based.
Poster presentation
Researching
I have to do a poster presentation, and I prefer to do it in LaTeX (as opposed to MS Powerpoint, Adobe InDesign/Pagemaker or Scribus) because I don't want to worry about formatting and I have existing code from LaTeX Beamer presentations and other tex files. beamerposter is probably the easiest after googling. To get started, I downloaded the beamerposter-example2.zip file, the latest beamerposter.sty, and beamerthemeI6dv.sty, and started hacking away. The institution logo is declared in beamerthemeI6dv.sty, so modify it.
Bash script auto-complete filename by file type/extension
Linux
Suppose I wrote a bash script with usage
script.sh filename.type
. That is, the script takes in filenames as arguments. Bash's tab completion works with all files. However, I've seen commands that will complete the filename even faster because it will select the file with the "right" extension. For example, pdflatex firstpart[Tab]
would automatically select firtpart.whatever.is.next.tex
. I'd like this feature for the scripts I write as well. Thanks to this post and this, I found out it is implemented through the complete
function in bash. For installed programs like pdflatex, the complete
commands are added to /etc/bash_completion
through the installer script (I think). For me, I can add them to my ~/.bashrc
:
RSS to twitter and mailing list (email)
Linux
In one of my group's website (operated using Google Sites), I post news and announcements to a page that has an RSS feed. I don't expect people to visit the website on a daily basis to find updates. With the RSS feed, updates can come to them using their RSS aggregator or reader of choice, such as Google Reader. However, it seems that in the internet-user population, only a subset uses RSS to get their information aggregated. Some uses twitter, and many uses email. I'd like to get the updates out to as many people as possible. However, I don't like to update each service by itself. My ideal workflow would be to send an email out with the news and the website, RSS feed, twitter, mailing list would be updated. Sending an email out that updates twitter and a mailing list would be easy. However, updating the website and RSS feed would be a lot more complicated, especially where you don't have access to the site's files via ftp or ssh (Google Sites is web-based).
Downloading, cURL vs. wget
Linux
I do more and more downloading using the command line these days, mainly using wget and cURL. This is a good comparison of the two. Both are great at downloading. cURL supports more protocol (outside of http, https, ftp) and is bi-directional. wget can download files recursively (links on a webpage, and links that appear on pages linked from that website, and on).
Wordpress and MySQL – error establishing a database connection
Linux
My server's been going down quite often recently (once a week). I believe it is a problem with the disk. Usually rebooting the server makes everything (including my blog) work again. Recently, I got the message "error establishing a database connection" even after the reboot. After many tries, this worked for me. Basically, my options table (prefix_options) was broken. Fixing it using phpMyAdmin worked.
Create bootable USB drives of Windows installation CD’s in Linux
Linux
Nearly all netbooks and many laptops don't come with an optical storage drive (DVD/CD). To re-install an OS, one would need to boot it from a USB drive. In this post, I outline how I could create a bootable usb drive for a Windows 7 disc. The information was taken from a reply on this post. I assume the usb drive is
/dev/sdb
on the Linux machine. ms-sys is required.
Add one to a number using Emacs’ regexp
Emacs
I recently had to add one to all numbers of the form
"n"
where n
is a number in a text file. Thanks to jlf
on #emacs
, I could do so with the the command replace-regexp
to change \"\([0-9]+\)\"
to "\,(1+ \#1)"
. I'm never going to remember this so I'm writing it down now.
Control my computer’s desktop (graphically) via VNC
Linux
Security
Since all my computers are Linux-based, I have OpenSSH installed on them so I can connect to them remotely. If I am not on the home network, I either have ports forwarded from the router or VPN to my home network in order to connect to my destination. In addition, I almost always use screen for all my terminal sessions. Thus, once I ssh to the computer remotely, I can resume my screen session.
UPDATE - Using my Playstation 3 (PS3) as a media player (play mkv files and more)
Hardware
Linux
Networking
I've been using mediatomb as a media server for UPnP devices such as the PS3. I definitely prefer XBMC to be on any device I use as a media player since I can play practically ANY media format (if the hardware can handle them, e.g., xbox can't handle HD content) and I can stream from local computers via a Samba mount. However, it is not currently available on the PS3 and it might NEVER be available; see this post. With recent advances in the PS3 Hacking scene, I'm sure the Linux OS can be installed on my PS3, and some advance media player will come into fruition (XBMC? GeeXbox?). In the mean time, I'm stuck with Sony's media player capabilities.
Update system BIOS on a Linux machine with a Windows/DOS updater
Linux
I recently had to update my BIOS on my Toshiba Portege R705 laptop. Unfortunately, the updater can only be run on a Windows/DOS system. Luckily, I remembered reading this article on Linux Journal that described how one can boot into FreeDOS to update the system BIOS. I will outline what I did.
Server Applications
Internet
Networking
I would like to compile a list of server applications that are available out there that I might use some day (or am using now). My preferences for software are GPL-based, lightweight, easy to setup up, easy to administer, easy to use, and widely-used (to get support). Most of these were found based on my experiences on the web or from searching.
Photo/picture/image repository, viewable using a web browser
Linux
I never had the habit of taking pictures and using them to cherish memories. I recently realized its importance when my mom passed away and I was scouring to find pictures. Luckily my siblings had some (I had to scan the the older, non-digital ones). These days, a camera is cheap, portable, and digital storage is cheap and only getting cheaper. I went out and bought a camera and will be taking pictures for the rest of my life now. I probably won't print the photos all that much, just viewing, sharing, and storing them.
Rip dvd’s (iso’s) to high quality avi
Linux
I recently needed to rip some videos from dvd's so I can have them on my computer in smaller files. I prefer the command line, so I use MEncoder that comes with mplayer. Supposedly AcidRip is a good interface that automates things for MEncoder. I wrote some scripts based on my research into how to do rip dvd's from the command line. Note that I have
-dvd-device /path/to/iso
in all of the scripts since I'm doing videos from an iso file. If you are doing it from disc, remove this portion of the commands.
Message of the Day (MOTD) at login
Linux
On Ubuntu 10.04, I like it how the the login message tells me system load, memory usage, and other useful statistics when I ssh into the machine. Found out that the script is known as
landscape-sysinfo
, and it is executed via the package update-motd
. Basically, you can add modify the MOTD by adding scripts that output messages into /etc/update-motd.d/
. See here for a more thorough explanation.
Mac OS X in Linux via VirtualBox (as guest OS)
Linux
Mac OS X
I recently installed a Windows virtual machine on my Linux laptop. I wanted to test out a virtualized Mac OS X machine via VirtualBox as well (just in case I ever needed to do something that can only be done on a Mac). My main references are this post and this post. Although they virtualize on a Windows host machine, it also worked on my Ubuntu 10.10 host laptop.
Virtualization with VirtualBox (running Windows inside Linux)
Linux
I recently re-installed Windows on an old laptop to use Adobe Acrobats and the likes. Some Windows programs just aren't accessible on Linux via Wine (although MS Office works). I've decided to set up Windows as a virtual machine on my Linux laptop; let's see if virtualization will take the separate Windows machine out of the picture. I went with VirtualBox since it is open source, free, and was mentioned on Linux Journal. My initial reaction is VERY favorable. It seems everything works flawlessly (internet, cd drive, accessing files from the host machine; usb supported only through PUEL proprietary version.). I'll jot down what I would like to be able to do with VirtualBox but have not been able to yet later in the post.
Slipstream service pack and create a new Windows XP installation CD
Uncategorized
I am not yet able to be Windows-free since some apps are only available on Windows and/or Mac OS X. Among those apps are those from Adobe: Acrobat and Pagemaker (or InDesign). I recently reinstalled Windows on an old laptop (I've actually started to use VirtualBox with Windows on my Linux laptop afterwards, but I will blog about that later) so that I can access those apps.
Create a video slideshow from images automatically
Linux
On a Mac, creating a video slideshow with some pictures is easy with iPhoto and iMovie. I haven't found similar programs on the Linux side (yet). I mean, there are many photo manager apps such as F-Spot and Shotwell that can create a quick slideshow from the selected pictures, but they don't export the slideshow into a video format that can be played anywhere (platform-indepedent). Also, they don't have nice transitions between pictures. digiKam supposedly supports these features, but I think they've been taken out.
emacs keybindings in firefox via KeySnail
Internet
Although conkeror is my default web browser, I do use Firefox and Chromium sometimes. Found KeySnail on the StumpWM wiki. It let's me define keybindings in Firefox (default setup has emacs-like keybindings). Now Firefox won't close when I accidentally press
C-q
(customized to "unfocus") and I can use "C-v" and "M-v" when paging through. Cool!
Be on my home network when I’m away from home via OpenVPN
Networking
Security
In my previous employments, I remember co-workers having to use VPN when they work from home. They can access everything at the company as if they were physically on-site. I haven't tried configuring it on my home network since if I ever needed anything, I ssh'd into my home NAS, and grabbed stuff from there. I guess VPN can be useful in that everything I do on the remote machine will seem like I'm at home, meaning all my mounted access to different directories on the NAS, access to the router, etc, are available while I'm away.
GNU General Public License (GPL)
Freedom
Linux
I never expected to be a programmer. However, due to the demands of doing data analysis on the computer and my general interest in computer software and Linux, I write code on a daily basis, whether they be in R, bash, python, or even C/C++. I would have to say I learn, use, and build software from what's already out there, in particular, open source software. Since others are generous to share their code, I of course am willing to share my code. Most of the code I run by are licensed under the GNU General Public License, so it's important for me to understand my rights when using GPL licensed software and when I release software under GPL.
Google Voice on Asterisk with an Auto-attendant and free calls
Networking
VoIP
I've heard about Asterisk for some time now since using VoIP services such as Google Voice, Gizmo5, Sipgate and Skype. Most of these services allow you to receive unlimited phone calls for free; you just need to register the service using a sip client, such as Ekiga on Linux, telephone on Mac OS X, sipdroid on an Android phone with unlimited data. The only "fancy" thing I did with these services was receive Google Voice calls on Gizmo5 and using the voicemail feature for parents in my local youth group to leave messages (so I don't have to release my personal phone number or talk to them…I am a busy person).
Creating even NICER, publishable, embeddable plots using tikzDevice in R for use with LaTeX
LaTeX
R
Researching
It's true. I like to do my work in R and write using LaTeX (well, I prefer to use org-mode for less formal writing and/or if I don't have to typeset a lot of math). I haven't done a lot of LaTeX'ing or Sweaving in the last year since 1) I've been collaborating with scientists (stuck using Word) and 2) my simulations in R have been a little overwhelming to keep in one file a la literate programming. I have a feeling I'll be going back to LaTeX soon since I have to write up my dissertation (and lectures if I end up at an academic institution, ****crosses finger****).
Emacs’ universal/prefix argument
Emacs
I've used
C-u
as a prefix to many key presses in emacs and conkeror before. I never really knew what it did. Recently, I read this that show you how to repeat commands by typing M-#
, where #
is a number such as 5 or 100, before a command, such as C-n
or 0
. I've also read function definitions in different .el
files or emacs' documentation that speaks of function arguments. I knew that emacs commands correspond to functions, but I never knew how to pass in arguments.
Finally started to version control my files using Git
Linux
I've been aware of different version control systems since I started using Linux. Why? Often times I am asked to install the latest development version of a software as the last tarball (*.tar.gz) has not been updated for a while. I've pulled code most often using Subversion (SVN), Git, and CVS. Of the various version control systems, I must say Subversion is the most prevalent out there. However, I've heard many preach about Git, similar to people preaching about Emacs (me being one of them). Even though Git is not as "standard" as Subversion, I chose Git because
Future of computing?
Internet
Linux
This is what I envision the future of computing to be if things were up to me. I can turn on any computer and be able to access an environment that has all the applications I use via a web login. That is, I should be able to remotely log in to my computer/server from any computer that has a web browser and be able to do things on it very fast. I also envision computers to be portable like that of an ipad, but with a keyboard peripheral that can connect to the device to turn it into a laptop (can't live without keyboard shortcuts; think emacs and conkeror).
S4 classes in R: printing function definition and getting help
R
I'm not very familiar with S4 classes and methods, but I assume it's the recommended way to write new packages since it is newer than S3; this of course is open to debate. I'll outline my experience of programming with S4 classes and methods in a later post, but in the mean time, I want to write down some notes on how to get help (via
?
in R) and getting function definitions from S4 methods.
Twitter in Pidgin
Linux
I liked Adium in Mac OS X because it allows me to use all IM services (AIM, facebook, etc.), IRC, and Twitter in one application. For some reason Pidgin (based on libpurple like Adium) did not have Twitter support natively. This site shows how, but it did not work. It appears pidgin-microblog in Ubuntu is deprecated. To get it to work, do
Wireless Bridge does not allow access to computers on the same network
Hardware
Networking
My primary router is on tomato, and since I've been liking it so much, I decided to load tomato on my wireless bridge. After doing so, I followed these instructions (Wireless Ethernet bridge mode) to it to connect to the primary router. It was way easier than doing so on DD-WRT, and access to the internet works. However, access to computers connected to the primary router did not work. I believe this is a known issue (read comments in the link).
Wireless Bridge on DD-WRT
Hardware
Linux
Networking
All my routers at home are either on the Tomato firmware or DD-WRT. I've set up wireless bridge many times but keep having to google it up as I keep forgetting the exact steps. I'm going to record it now for my reference. The following is taken from here, the V24 build instructions.
softmod original xbox + xbmc
Hardware
Linux
Networking
Now that I have a NAS set up as my file repository, I would like to access my media on the TV. I bought an original xbox (1) on craig's list for $20, the xbox dvd remote on amazon for ~$6, an xbox usb connector on ebay for ~$2, and the original Mech Assault game at Game Stop for $2. I also needed a compatible usb drive, which I obtained by borrowing my sister's Sandisk Cruzer 1gb flash drive.
Migrating back to Linux (Ubuntu) from Mac OS X
Linux
Mac OS X
After abour 1.5 year with Mac OS X on my beautiful black Macbook (v 4,1), I've decided to migrate back to Linux, namely Ubuntu 10.04 LTS. Why? Well, I was actually using Kubuntu (KDE version of Ubuntu) from about Spring 2006 to November 2008. At the time, I remember setting up Kubuntu on my Dell Inspiron 700m laptop was quite a task: getting the display to work at the right resolution, wireless, etc. Once everything was set up I loved the environment. Afew things bothered me then: no MS Office (OpenOffice doesn't cut it sometimes when you have to collaborate), graphics environment (KDE) crashed from time to time (probably because laptop was too slow and I multi-tasked daily) and lack of support for third-party hardware (syncing Palm Centro and ipod, scanners, etc). I would have to do lots of research to figure out the third party issue, and often, a solution was not available (maybe arriving much later). I kind of got sick of this whole process so decided to switch to Mac OS X since it was Unix-based (all tools available), had MS Office, pretty, stable, nicely built (quality), and had third-party hardware support.
Stream music to client via DAAP
Linux
Mac OS X
Networking
Now that I have a NAS at home, I plan to place stuff that takes up a lot of space that doesn't get used so often on there. However, with that comes the issue of file access. Since my NAS is linux based – ubuntu server edition to be exact – I can access files via ssh, sftp, ftp, sshfs (map the drive), samba, etc. I should probably write a blog post about sshfs and samba soon.
Multiple Dropbox instances on UNIX systems (Linux, Mac OS X)
Linux
Mac OS X
I have one main Dropbox that I use and try to grow by referrals. I use it mainly for files I feel are important (backup!) and I need accessible everywhere. Sometimes I need to collaborate with others or have additional files I need accessible everywhere (for a short period of time). I don't want these files filling up my main account so I have side accounts (2gb). To get multiple instances on a Mac OS X I followed these instructions. For other Unix-like machines follow these instructions (includes a link for Windows too). NOTE: Don't follow the Linux instructions on the first link.
Accessing MS SQL Server from Mac OS X (or Linux): FreeTDS, unixODBC, RODBC, R
Linux
Mac OS X
This post is a followup to a recent post I wrote. The issue at hand is that data are stored on a SQL server (Microsoft Server 2005 or version 9 of MS SQL Server to be exact) and that I would like to send queries from a command line interface, either from Mac OS X or a Linux system. The data queried from the server will most often be analyzed in R.
repeater bridge with DD-WRT (Linksys WRT160Nv3)
Hardware
Networking
[gallery] My Trendnet 652BRP primary router (DD-WRT) with a Linksys WRT160Nv3 (DD-WRT) as a wireless bridge did not work too well (setup here). Sometimes my laptop disconnects to the wireless network and I have to unplug/re-plug the primary router to be able to reconnect. This occurs fairly often (2x a week?). I don't know what the deal was, so I decided to swap the routers as primary and secondary. In addition, I set the Trendnet router as a repeater bridge as opposed to my previous setup. Instructions can be found here. The instructions worked well, but I struggled with a few points:
Backup your gmail account in Linux/Unix or Mac OS X using getmail
Linux
Mac OS X
[gallery] Even though I have utmost trust in Google's data centers (>= 99.9% uptime), I finally decided to backup my gmail account. I am pretty OCD when it comes to backing up my data (bi-weekly backups of my home directory into 3 external hard drives, one of which is in a water/fire-proof safe box; MOST important research-related files in the cloud via dropbox), but I've never backed up my mail archive which holds a lot of valuable information.
Wordpress: post via email using Postie
Internet
Wordpress's default "post by email" setting is pretty weak, mainly because it doesn't handle attachments (think posterous, among other things. Postie is quite powerful as it can handle different attachments and can handle my LaTeX and source code content. Note to self: use the development version!
Using Google My Maps: planning my stay in San Francisco
Internet
Researching
So I have to be in SF for an internship this summer. As with all relocation there is the hassle of finding a place to live. After googling to find a suitable apartment, I realized that Craig's list is the best place to go. I tried to avoid it at first since I just wanted to deal with apartment managers, but I realized that many apartment managements are bad (the reviews!). I thus resorted to Craig's List (temporary and sublets) and housing options information from UCSF, such as this. Finding a place to live is a bitch because I won't have a car (rely on public transportation). Thus I have to get a place that is close to public transportation stops (thank goodness public transportation in SF is great), especially close to where the company-sponsored bus stops are at. After finding an ad, looking up a location, weigh in a bunch of information (furnished, rent, utilities, etc), comparing it to bus stops, things got out of hand. I needed a program/service that plot out multiple locations. Google Maps came to the rescue! I used the My Maps feature (click on it), created a private map, plot out where the bus stops (company) are, 24 hour fitness, gyms, costco, etc. Next, each potential housing option, I saved it, along with it's link (eg, craigslist) and additioinal information. This made the job a little easier (housing hunting is always a bitch!). Be sure to google or find out where the bad neighborhoods are, eg, tenderloin in SF.
Teaching with a graphic tablet and a projector: replacing a whiteboard or overhead projector
Mac OS X
Teaching
Many times when I teach, I often want to keep a copy of what I write on the board. Why? I can have a softcopy to refer to when students have follow-up questions, I can re-use these notes for the next discussion if there are only a few people, and I can re-use them in later quarters. In high school, I always liked the overhead projectors. Teachers can write and look get feedbacks from the students face to face. Also, you can just scroll your super long sheet up as you write and not worry about erasing like on the white board. Later in lecture, you can pull them back if you want need to refer to the previous notes. I always had an idea to use some kind of a tablet device with the projector, which all modern classrooms should have. I finally decided to buy a fairly inexpensive Genius graphic tablet from Walmart for $40. Wacom tablets are probably better, but they are very expensive, especially the larger ones. I guess they are expensive because they are meant to be used with photoshop by artists. In my case, I need a large tablet but it doesn't have to be premium since all I'll be doing is writing words and math symbols on it. My platform is a Mac OS X, and this link discusses some tablets and programs. This site discusses the tablet PC's use in the classroom altogether. After some trials, the best program seems to be inkbook, which costs $40. However, I guess the Genius driver doesn't play well with inkbook. To get it to work, I have to turn on "Handwriting Recognition" in System Preferences > Ink. In Inkbook, i can write in the ink mode. I see a yellowpad whenever I write. This SHOULD NOT be there, but it works for now. Wacom would play nicer. Update 3/27/2010: I returned the Genius tablet a few weeks ago as it does not play nice with inkbook. The author recommended a Wacom. However, it is too expensive for me (a decent sized one) and I won't need one yet since I don't need to TA next quarter. Hopefully, the Apple Ipad will have a pen with it so I can use it with this (main reason I would buy one). Otherwise I would have to consider getting a cheap netbook with a pen.
Linksys WRT160Nv3 with DD-WRT as wireless bridge
Hardware
Networking
I recently bought a refurbished linksys wrt160n-rm from amazon for $31. I ordered it before getting DD-WRT working on my trendnet router. I didn't want to hassle with returning it, so I decided to turn it into a wireless bridge, something I wanted to do for a while. With the refurbished router, I had to insert the refurbished instructions cd in order to get the router working. Once the linksys firmware is on, I opted to get dd-wrt working on it. I followed the instructions here, but couldn't get things working because the 30-30-30 reset made the router not work; I had to unplug and plug the power back in to get an ip again. Also, the firmware on the page at the time did not work when i tried to upgrade without the 30-30-30 reset. Googled a bit, and postings here mentioned that they got dd-wrt working without the 30-30-30 reset and with an earlier build. I did so, and got dd-wrt working. Note that I also checked this page to confirm that the other build was required first. Once this version was installed, I upgraded to this version. To get it as a wireless bridge, I followed the instructions here. Voila, I now have the wireless bridge in my brother's room with his computer and our network printer connected. Things are sweet! In the future, I would probably have a router in the living room connected via a network cable to act as another hotspot for my wireless network or something.
Trendnet TEW-652BRP with DD-WRT firmware
Hardware
Networking
So I own a relatively inexpensive trendnet router (on sale ~ $25) for a while now, and I've been trying to get DD-WRT working with it since having it. This site says it is supported, but I couldn't find a firmware available until now. This post (page 16) has a link to this site that host some binary firmwares. Using the web interface I flashed the firmware successfully and now has a powerful router with DD-WRT. Cool! I use the dd-wrt to assign my laptop a static ip for port forwarding, among other things.
Scheduled Parallel Computing with R: R + Rmpi + OpenMPI + Sun Grid Engine (SGE)
R
Recently I've learned how to do parallel computing in R on a cluster of machines thanks to the R packages snowfall, snow, and Rmpi. I've been using the SOCKET method with snowfall since together they make things simple. With these tools, I can reduce day/week long jobs to hours or a day across many (100) cores/cpus.
vcasmo: video/presentation + picture/slides together!
Internet
I was reading this blog post on Revolutions Computing's blog and discovered the vcasmo service, a way to post a presentation or video with the slides right next to it. This is DEFINITELY something I will use. Often times I watch talks or presentation videos online and most of the time the videos are either focused on the speaker and the projected slides or on the presentation slides with voice-over from the speaker. This method of presentation highlights both the presentation slides and what the speaker has to say, making me feel that I didn't miss much from not being at the talk physically. Here is the embedded talk that made me discover vcasmo:
email yourself when a script or a job is done
Linux
Mac OS X
R
i usually run long simulations in R on a remote server. i've read an article on linux journal about emailing yourself after something happens…i forgot. since my jobs are long, and i usually have to run multiple jobs sequentially, it'd be nice to look at my results immediately when they're gone. good thing *nix based systems have the mail (or mailx) command:
file management: emacs dired to replace Finder in mac os x (and other OS)
Emacs
Linux
Mac OS X
I hate Finder in Mac OS X. I mean, it looks nice and all, but it is not customizable. I really liked the KDE window management back when I used Kubuntu because I can select/de-select files with the keyboard, cut/paste/copy files with the keyboard, have shortcuts to different locations, etc. Finder has some keyboard shortcuts, but not at all flexible. You can copy and paste files, but you can't cut and paste files. Selecting/de-selecting files is not as flexible (eg, skipping a file). I can't go to the sidebar window where all my bookmarked locations reside using the keyboard. Finder, and in general Mac OS X, is not so flexible. This bugs me a lot. I wanted to get stumpwm working in OS X so that things are more keyboard-oriented. However, it doesnt work with all the native mac apps. The fix is with dired mode in emacs. I spend most of my time in emacs already. With dired, I can even manage files inside emacs (I can do so in the shell, but this makes browsing and executing commands on files a breeze). Use "C-x f" and navigate to a location rather than a file. Learn all the keyboard shortcuts for delete, rename, execute shell command, etc. You can even cut/copy/paste with wuxch-dired-copy-paste. The functions below also allow me to open files the Mac way, with the "open" command. This is cool because files will be opened with the native Mac default application.
project management and task management with Google Apps and Many Moon
Internet
so i spoke about open source crm services and programs before. google really made it easy for groups to collaborate. i can attest because i use it with my youth group. Many Moon came out that made project management and task management even easier and better with Google Apps (well, i don't even think u need this, u just need email probably). I haven't used this really yet, because i don't know if my youth group will be willing to use it. they don't like the feeling of "work." i, however, love to be efficient and productive. this service really makes any small business or group efficient and productive like the big corporations. great one!
online scheduling and calendar sharing: tungle
Internet
Tungle is a great service for calendaring, no matter what platform u use (google calendar, exchange, ical, etc). it lets the person that wants to schedule a meeting with u view your available times, they propose times, and u pick the time u want. this definitely helps when ur busy. oh and its free, and only requires u to have email. best of all, once u select the time, ur calendar automatically gets updated.
open and save read and write protected files in emacs (sudo in emacs)
Emacs
Linux
Mac OS X
Often times i have emacs open, and would like to open and edit protected files. since i didn't launch emacs as a root user via sudo, i won't be able to do anything but close emacs and relaunch or use sudo vi on the file. googled it, this post shows how one can use tramp to open and edit those files with an existing emacs process.
emacs key bindings in mac os x (cocoa)
Coding
Emacs
Mac OS X
so im very used to emacs keybindings. i use it for emacs and conkeror so often times i hit M-w etc and these keys don't work in mac os x. this sucks. i googled it, and found this and this. guess u can have emacs keybindings in mac os x…for programs that use the cocoa appkit. too bad this doesn't work when i edit in neooffice or ms office.
free webinar web app: dimdim and drop.io
Internet
for free conference call i've been using Free Conference Call. not bad for a free service. however, it is phone only. what if i want to share my screen? from this list, i found dimdim. dimdim is free for up to 20 people. plus, no need for additional software – u just need a web browser with flash and install a 2mb plugin. they also provide a dial-in number…but i guess i should stick with free conference call if i want to keep the same passcode for all conferences.
Group messaging: automated phone calls and text
Internet
so i've searched many times before how to make automated phone calls for free, but found no real solution. i've accepted the fact that some things will not be free, so i started looking into paid services. found a free/paid service that's looking good called DialMyCalls. It gives me one free call to 25 people a day (with ads along with that voice message, before and after), and $.06 per 30 second message. this will do for now. found this post that gives a code for 50 free credits when i sign up: D381735.
Compiling and Cross-compiling R packages for Windows (win32)
Coding
Linux
Mac OS X
R
so recently i'm learning how to compile and cross-compile R packages from source for windows machines, which means i have to create windows binaries. the first option is to build on a windows system, and the 2nd is to cross-build on an intel-based linux or mac system. i will outline my experience.
Perfect VoIP combo: Google Voice + Gizmo
Internet
VoIP
So I've had my google voice telephone number for quite a while now, but i haven't used it much since there is no good google voice app for my windows mobile phone. i'm still waiting for sprint to release the new android-based samsung phone. i'm sure google voie would go VERY well with the google voice service.
hacking/editing portfile (MacPorts) to compile madplay for use with shell-fm (last.fm)
Coding
so recently i compiled shell-fm, a command-line last.fm player. in order to record music with shell-fm, i can output the stream of incoming data into tee, which saves the data to a file and at the same time pipes it out to stdout, and have that piped to madplay, a command-line player.
record music from last.fm using shell-fm
Coding
as i mentioned before again and again, i've been working on to get shell-fm to record music from last.fm. originally, i was trying to get shell-fm pipe to tee to save the songs and play using madplay (or mplayer) based on this method. this worked. i just need to write scripts to post process the songs to get id3 tags, etc. things work, and i have something like this in my shell-fm.rc file:
using pastebins for irc: pastie.org
Coding
Emacs
Internet
so i've been using erc for irc these days to get help on compiling stuff, eg, shell-fm and madplay. i usually just paste the errors i get which sometimes can be long. however, that is bad courtesy in the irc channels as that junk takes up a lot of room. jkramer from #shell-fm on the freenode told me to use a pastebin, googled it, and realized what i've been doing was very rude. so he sent me to pastie, and now i learned something new. paste my stuff there and send the links. pastie is a simple pastbin service, just paste. it also has a bot u can paste too, which is even more convenient.
an innovative idea? blogging by email: posterous
Internet
so i recently discovered (through michael zeller again) a new blogging service called posterous. it's pretty innovative in that u can post by email. really? blogger let's me do that too, and so does wordpress and most blogging services. however, posterous stands out because it'll do all the work for u. put in a link to a video and it'll embed it on your blog. attach and song, same thing. attach multiple pictures and it'll embed a gallery. want to format ur blog? write the html and posterous will take care of everything. cool huh? look at the list of features yourself. i'm still debating on whether to switch over or keep using blogger. i mean i guess i can start another blog on there for a different topic, but face it, i only have time for one active blog right now, and that i don't have another interest that i could blog about. to switch or not to switch? some drawbacks i see with posterous:
make java work in conkeror (xulrunner) in mac os x
Mac OS X
so i recently tried screenr and screenjelly to make screencasts. one problem, the java didn't load in my conkeror. it says i need to to install it. however, when i tried them in firefox and safari, things were working. must mean conkeror isn't finding it. emailed the conkeror list, got one feedback, googled for help. after some research, i found a solution that works:
youtube alternative: vimeo and blip.tv
Internet
a few months back, i uploaded a slideshow onto youtube. got an email from them that there is copyrighted material (the songs). then, youtube stripped the audio. this is rediculous. i googled for other alternatives and found a site that compared different services. vimeo and blip.tv seemed to be the best. i uploaded my video to vimeo and it's still great. i chose vimeo because i've seen other bloggers (emacs, linux, etc) use vimeo. i usually agree with whatever this crowd uses.
web app to make screencasts: screenr + screenjelly
Internet
so these days everyone is making videos, especially tutorials on their computer. screenr and screenjelly are two services that allow you to make the screencasts easily. u go to their website, load their java app, select the screen space u want to record, and hit record. the first service gives u up to 5 minutes, and the 2nd to 3 minutes, plenty of time for a typical screencast. anything longer and u run into the danger of boring your audience.
Cool articles in the New York Time’s: Statistics + R
R
Statistics
so these articles are 'old news,' but here i am to blog it down before i forget. First article is entitled "For Today's Graduate, Just One Word - Statistics," and the second article is entitled "Data Analysts Captivated by R’s Power."
Free online radio: last.fm + pandora
Internet
so i found out about online radio through michael zeller (again). i did do a search for this back then, and i thought what i wanted were podcasts….radio through itunes. however, googling it i realized that wasn't what i wanted. podcasts are good for tutorials or talk shows or the news, but probably not for music.
surf web anonymously using tor
Internet
I looked into this as a kid when i first got into the internet, but didn't do much with these proxies because ur connection becomes slow. However, based on this post, if i reach a site that won't let me get access because of my location, then using a service like this should allow me to get to the site.
global environment variables (like PATH) in mac os x via environment.plist and /etc/launchd.conf
Emacs
Mac OS X
so, through my journey of getting emacs 23 and conkeror to work on mac os x, i've discovered that PATH is not defined for native mac gui apps. details of the journey can be found in this post of mine's.
Online storage: A Drive
Internet
So I mentioned before about using Drop Box to sync my files, and I use google sites to host my website, etc. However, both have limits on files uploaded and stored. What if I have large media files? I can upload videos to youtube for videos, but they convert the video to flash, and these days they remove audio if u have any copyright sound like music. Crap.
debugging errors in emacs (.emacs)
Emacs
So recently i wanted to try to add on other popular emacs packages such as elscreen and Do Re Mi (doremi). However, I couldn't configure them to work as i got some errors after adding stuff to my .emacs. I posted comments on people's blogs and emailed them to try to figure this out.
Some recent emacs package that I started using: windmove, ido, elscreen, doremi, uniquefy, doc-view, window resize, eval-region
Emacs
Make sure tha path to the packages are added via add-to-list. Also, as mentioned in the previous post on debug, doremi and elscreen should be towards the beginning of .emacs. Doremi conflicts with multi-term: (require 'multi-term) (multi-term-keystroke-setup) (setq multi-term-program "/bin/bash")
org-mode: outlines & meeting minutes + others (create / export html)
Emacs
So I've recently discovered org-mode for emacs. I don't need to put much references up as a simple google search for org-mode should yield many references, especially its home page (there, u can find many references to start using org-mode).
Generate multiple forms and filling them with data from a spreadsheet or database
Internet
Suppose I have a form (template) that I want to fill in, and I have a database or spreadsheet that has all the data. I would like to generate forms that are filled-out using records from a data source. I want to do this automatically of course.
Backup my google calendars via shell and python scripts and scheduling them via cron
Internet
So recently my calendar got messed up a little (dupes) due my mobile phone (HTC Touch Pro running Windows Mobile 6.1) resetting [syncing again messed it up]. I realize (now???) I need to keep backups of my calendars and contacts.
emacs-style web browser: Conkeror
Emacs
I remember back then when I used Kubuntu, I love Konqueror because I can hit
Ctrl
and all the links would be highlighted. I just need to type the letters corresponding to the link and Konqueror would take me to that link. I love it because I didnt have to use my mouse or touchpad. Recently, I discovered that Firefox had a similar feature, and it made me love firefox even more.
Parallel computing in R: snowfall/snow
R
I finally found the time to try parallel computing in R using snowfall/snow thanks to this article in the first issue of the R Journal (replacement of R News). I didn't try parallel computing before because I didn't have a good toy example, and it seemed like a steep learning curve. Snow and Snowfall is perfect for 'embarrassingly parallel' jobs, eg, a simulation study, bootstrap, or a cross-validation. I do simulation studies a lot, eg, assessing the properties of a statistical methodology, so implementing parallel computing will be very useful.
passwordless ssh
Linux
Mac OS X
I can't believe I never set this up. I ssh to a few of my school servers all the time. I always have to enter my password to authenticate. I just found out I could authenticate with a key file instead of entering my password. I found the information from here and here. To set up, execute:
Open remote file while in emacs ansi-term buffer/window: ansi-term + tramp
Emacs
In emacs, I can edit files remotely using tramp. While ssh'd to a remote server in ansi-term at a specific location, I can open the remote files in emacs as if that remote location is my working directory. This is taken form here. Put the following in the remote server's .bashrc file:
edit files remotely: emacs + tramp
Emacs
Linux
Mac OS X
Suppose I want to edit a file remotely. I don't want to download/ftp the file to my computer, edit, and send it back to the remote server. In emacs, I can edit it remotely using tramp via the ssh or rcp protocol. Put following in the
.emacs
file after installing tramp.
Run screen in emacs with ansi-term (combine this with emacs + ess + remote R)
ESS
Emacs
This is actually an update to this post, but since I discovered a few more things, I'll write a new post. To run
screen
within a shell buffer in emacs, I tried M-x shell
and fired up screen
(ditto with M-x term
). It gave me this error: Clear screen capability required
. I found the solution to this here. To fix this issue, do M-x ansi-term
(use /bin/bash
when asked of course). screen
now works inside emacs. Combine this with running a remote R session in emacs, and there you have it, the perfect work flow for developing and running computationally intensive R code! I can utilize screen
to not have my R sessions interrupted, and I can utilize ESS to send code to an R session/buffer. I have to say, this WILL be the way I use R for any computationally-intensive project!
LaTeX in blogger, pt 2
LaTeX
In my previous post on this topic, I didn't get LaTeX to work in Blogger because forkosh closed their mimetex service to the public. For LaTeX to work in blogs, I would either have to switch to wordpress or get my own host and install mimetex. The First option wasn't too appealing as I'd like to keep everything google since a lot of my personal services are hosted here (yes, I'm not afraid of google having too much information about myself). Second option also wasn't feasible. I found out from some more searching that codecogs is generous enough to host this kind of service. I updated wolverine's script in firefox/greasemonkey with this, and now I have an UnLaTeX button as well! Really cool. To use, in compose mode in blogger, type dollar sign dollar sign LaTeX code dollar sign dollar sign, then hit the latex button. Bamm! To see original code, hit UnLatex. Here is an example.
Getting information and learning: google + wikipedia + wikibooks + wolfram alpha
Researching
Teaching
When I want to find information, Google is my #1 resource. The syntax trick I use most which I learned in the research writing class (Writing 39C) at UCI is "The first word is the most important" (the first word narrows the search!). Some other tricks are "- Don'tWantThisWord" and ' "this quote exactly" .' Google usually leads me to Wikipedia, an online collaborative encyclopedia, or Wikibooks, an online collection of collaborative textbooks, for my answers. These three resources are where a lot of my learning/review of statistics are found.
Scribd: embed documents in your blog/website
Internet
Want to share a document online? Whether it's a MS Office doc (xls, doc, ppt), open office document, txt/rtf, or a pdf, Scribd allows users to upload a file and have it viewable on a website by embedding their iPaper program (Flash-based). This is cool as visitors no longer need specialized software on their computer to view the document.
Collaborations via Google Docs + Compilation of data with Google Docs Forms
Internet
Google Docs has been around for quite some time. You can view and edit word docs, spreadsheets, and presentations online. You can also view pdf files. Besides from the abilities to view and edit documents, collaboration (multiple users editing the same file) is also possible. Instead of sending files back and forth between users, we can invite collaborators via email to have access to a Google Docs file (must have google account). Then, any collaborator can edit the file and everyone else can view the changes made the file is opened (or immediately in real time if it is currently open). This is especially useful for small businesses and non-profit organizations (eg, my local church youth group).
Create multiple graphics in R without multiple calls to pdf / postscript / jpeg / png
R
To save multiple graphics, e.g,
Rplot001.pdf
, Rplot002.pdf
, …, Rplot050.pdf
, we don't have to call pdf()
50 times (or another graphics device function) in R. Instead, use Rplot%03d.pdf
for the filename
argument in pdf()
and each plot()
call will be saved to a new pdf
file. Run dev.off()
once at the end to close all devices!
Editing/adding on to Sweave features in ESS
ESS
R
I really like David Whiting's Sweave methods in ESS:
M-n s
M-n P
and I get a compiled pdf file. However, i wanted to change it so Sweave uses the cacheSweaveDriver() when sweaving. I knew the the functions were defined in the ess-swv.el file (in my carbon emacs in Mac OS X, it is at /Applications/Emacs.app/Contents/Resources/site-lisp/ess/ess-swv.el). Now, I edited the file to create a new way to Sweave:
rubber - latex’ing multiple times to get the final product
LaTeX
Linux
Rubber is a Python script that assists with the compilation of LaTeX documents. It is useful when you use
bibtex
for bibliographies because multiple latex
and bibtex
commands are needed to be executed for the document to be generated. With rubber
, I can issue a single command and the necessary commands will be executed automatically:
No matching items