/home homepages

August 16, 2011

www.cs.miami.edu/~[username] is now supplemented with www.cs.miami.edu/home/[username].

This change allows [username]‘s homepages to be addressed by both style URL’s.

Your account was created with a symbolic self-link “public_html->.” in you public_html directory. This self-link means that the new URL scheme will work along side the old URL scheme without any change to your homepages.

If one or the other URL is not working, it is easy enough to fix.

blog.cs.miami.edu is moving …

August 3, 2011

WordPress for the department is moving to a new computer. Using the new “Network Admin” possibilities of WordPress 3.0. It is a big improvement.

Some tricks for wordpress users:

To move a site, it helps to change the url’s in the database:

mysql> update wp_options set option_value='http://jackson.cs.miami.edu/burt' where option_name='siteurl';
Query OK, 1 row affected (0.00 sec)
Rows matched: 1 Changed: 1 Warnings: 0

mysql> update wp_options set option_value='http://jackson.cs.miami.edu/burt' where option_name='home';
Query OK, 1 row affected (0.00 sec)
Rows matched: 1 Changed: 1 Warnings: 0

If you don’t do this, you get some pages to work, but then it will jump to the URL as described by these values. This worked pretty good on the old WordPress, and might work on WordPress 3.0 without Network Admin enabled. I couldn’t migrate the URL of a 3.0 with Network Admin and ended up dropping the entire database and starting again.

In general, it is hard to change the URL of a wordpress site. When migrating a site there just has to be downtime – export the data and shut down the old site; fix the DNS and install at the new machine on the desired URL.

To set the password, find a working user_pass from another blog, with known password:

mysql> update wp_users set user_pass='$P$BV3q/rYU98poi/Zv/ARuStuPi.Wvad.' where user_login='admin';
Query OK, 1 row affected (0.02 sec)
Rows matched: 1 Changed: 1 Warnings: 0

Subversion moved

July 28, 2011

The subversion server has been moved to a new machine. The CNAME svn.cs.miami.edu will lead you to it. It’s actual name is meade-vma.

Changed is the network service style. Previously the repositories were accessed using webDAV and Apache, and the repository URL used http:// as the prefix.

Now svnserve is being used, and the URL should be svn://svn.cs.miami.edu/[path-name].

Route53 live

July 28, 2011

Route 53 went live as of June 26. There are now two infrastructures for the department’s DNS servers for forward resolution (that is, from name to IP address). We maintain the classic BIND named’s, housed in the department with on-campus secondaries to cover short term outage and maintenance; and a parallel cloud DNS service that can withstand wide spread catastrophe.

cs.miami.edu on Route53

June 27, 2011

Why the change:

Until the summer of 2011, our DNS servers were diversified across the campus. This meant we were vulnerable to an entire-campus network outage. Until recently, that risk was acceptable. In particular, with our email servers on campus, a campus wide outage would stop our email as well – a much more serious concern operationally.

The department is experimenting with the cloud-based email service Google Apps. With cloud-based email, we are now in a position to demand higher availability, and we can build towards continued email functionality even during an entire-campus network outage. We need therefore to raise the bar on DNS availability.

What is changed:

To match cloud-level availability of email, we will join a cloud-base DNS service, namely Amazon Route53. This is paying service, based on usage, but I expect yearly costs to remain under $20.

Second, our existing network of DNS servers must be limited to DNS servers which we have administrative control. The primary-secondary discipline of a network of DNS servers is such that updates to the DNS information must originate at the one primary server and are propagated automatically to secondary servers. In the event of the loss of the primary server, the secondary servers cannot be updated by the primary-secondary approach.

During a prolonged outage, it might be necessary to update the DNS information: in particular, the update of MX records to redirect email towards a cloud-based email. Hence indiscriminately propagating DNS information to multiple secondaries can paradoxically lower availability by freezing outdated information.

Domain Zones Affected:

The department has both internal and external DNS zones. All internal zones are unaffected by this project; and only one of the external DNS zones is affected. The external zones are cs.miami.edu and 89.31.192.in-addr.arpa. Only cs.miami.edu needs to be treated, as the reverse lookup domain is not obligatory for most functionality.

Since the internal domains are unaffected, no new administrative procedures are needed for internal domains.

The external domains will continue to be administrated as before, using Named, with new procedures for and isolated to Route53 administration. Route53 does not use the primary-secondary discipline. Hence, the new procedures will require individual updates to Route53 and Named.


Route53 is a cloud-deployed web service. It has two interfaces. It is queried using Bind (or tools such as dig) and that is how it is part of the DNS infrastructure. The other interface are XML documents submitted by https to an the Amazon URL https://route53.amazonaws.com/2011-05-05/{service}, where {service} selects for the service desired.

See: http://docs.amazonwebservices.com/Route53/latest/APIReference/

Route53 requires and Amazon Web Services (AWS) account. An account has been opened with the mail address amz-dns@cs.miami.edu. This AWS account has a associated with it an Access Key ID and a Secret Access Key. These are needed in the web requests as a sort of username/password pair. Although https is used to protect privacy, the Secret Access Key is never divulged in a straight forward manner, but is used to MAC sign requests.

This account is where billing occurs as well. The email amz-dns@cs.miami.edu would be used for support with Amazon for this product. We will share the password to this account. However, administration of the DNS database needs only the Access Key ID and Secret Access Key, and does not need to log onto the AWS account.

Updating RR on Route53:

Various tools exist to manipulate the http interface. R53Fox is a graphical front end that allows inspection of the zone database and some limited changes.

The workhorse utility is the Perl script dnscurl. Dnscurl prepares the command line and executes curl including the special headers that authenticates the request, allowing change access to our zone data. Most changes are a POST of an xml file describing the change.

See: http://docs.amazonwebservices.com/Route53/latest/DeveloperGuide/

Dnscurl finds the AWS account Access Key ID and Secret Access Key in the file .aws-secrets. This file must be kept confidential.

Route53 XML Requests:

Route53 uses xml. A GET request to a domain will return the information in xml. To delete, create, or change RR’s requires preparing an xml document and using dnscurl to POST the document. The document returned by the GET is useful to fashion the document to POST.

A change is done by deleting the RR’s and the creating the updated RR’s. The POST works as a transaction. A document with multiple changes either succeeds entirely entirely or fails entirely. Therefore a change, which seems like a delete followed by a create, to the outside world seems like an update – it is impossible for the deletion to succeed but the create fail.

It is possible to hand edit the xml documents returned by a GET request, and submit these documents using dnscurl. It is easier to use the R53Fox GUI tool. Since this tool is built on Firefox technology, XUL and Javascript, it should run on any platform that has an xulrunner.

Triple work:

There are now three sources of DNS for cs.miami.edu. There is the primary/secondary public DNS hosted at the department, with various secondaries on campus. There is Route53, outsourced DNS that must be held synchronous with the on campus source. And there is the internal DNS, which has different information. In some cases, all three must be changed, and each has its own method to change.

An example where all three have to change is www.cs.miami.edu. This is a public RR with A record value and is in the databases of by Route53 and the primary/secondary DNS. To update it in Route53 use R53Fox. Then log into pickett and change the master file and HUP named, remembering to increment the serial number so it can be picked up by secondaries.

We also use the external address internally. Even though www.cs.miami.edu is pickett, with internal IP, we want internal clients to browse www.cs.miami.edu the same as external clients. Therefore the internal database needs updating. Since we run dynamic DNS internally, to update use nsupdate on mcclellan.

[burt@mcclellan}: nsupdate
> update add www.cs.miami.edu. 86400 IN A
> send
> quit

Route53 DNS service

June 25, 2011

We are testing using Route53, a cloud-provisioned DNS service from Amazon, for a secondary DNS system.

Power outage

August 25, 2010

The Ungar building suffered a power outage. Our web servers were out for an hour or so, to avoid overheating. Sorry. Back now.

Proxied HTTP

May 29, 2010

Ssh has a thing called DynamicForward. It is actually SOCKSv5. Since Firefox is socks enabled, you can use your cs account as a proxy for web browsing. This is useful, for instance, to get to web pages that are only available from inside UM, such as CCS, or some of our internal-only sites.

  1. In .ssh/config include the line “DynamicForward 1080″ or use -D 1080 as an ssh option on the command line.
  2. In Firefox->Preferences->Advanced->Network->Settings find the panel “Configure Proxies to Access the Internet”. Select Manual and for “SOCKS Host” enter and port 1080. Selection also “SOCKS v5″. Then OK.
  3. To remove the proxy, select “No proxy” from the “Configure Proxies …” panel.
  4. No need to restart browser. You have to have ssh up and running.

What happens is that all HTTP requests are sent to port 1080 on your localhost, wrapped inside of a SOCK protocol. That request goes into the ssh session and emerges on whatever machine you are ssh-ing to, say Lee. Lee makes the HTTP request in proxy for your browser, and returns the response back through the ssh channel´╗┐.

Minicourse in Unix and HPC

April 22, 2010

Information posted here.

Subversion opened for classes

April 7, 2010

Subversion server going on line at svn.cs.miami.edu. An experiment with use in class at the unofficial name web.cs.miami.edu/svn/repos. Authentication is an unsettled issue.

Powered by Wordpress and MySQL. Theme by Shlomi Noach, openark.org