## Thursday, April 13, 2017

### Half the reason for POMDP's in the literature: You can pronounce the name!

Today, I noticed a new paper in Mathematics of Operations Research, which suggests a way to combine risk-sensitivity with POMDP's without having to use only exponential utility functionsI'm bothering to write this post because the title is infuriating to me. In my opinion, the title of the paper will probably largely contribute to why it will be forgotten – only to be referenced by other papers that find it while doing their post hoc due diligence

"Partially Observable Risk-Sensitive Markov Decision Processes"
by Bäauerle and Rieder
Mathematics of Operations Research (2017), Articles in advance

The great success of POMDP as a framework used in the literature is in part because people like the name – "POM-D-P". If you're taking in a young graduate student, you can send them to the literature with a few quick words – "Go check out POMDP's".

But what am I going to do with PORSMDP? "Poor-Sim-Dop"? "Po-Rism-Dip"? Even if one of those managed to roll of off the tongue, the relationship to POMDP wouldn't be obvious.

Why didn't anyone in the chain of custody of this manuscript suggest putting "Risk-Sensitive" up front, as in RSPOMDP (R-S-POM-D-P)? Or maybe at the end, as in POMDPRS ("POM-DiPpeRS")? The latter suggestion not only is memorable, but it sounds delicious.

Just some food for thought.

## Tuesday, March 22, 2016

### Phone scam: "Enforcement action" from "US Treasury" (302-279-3069)

At 8:00 AM this morning, I received a phone call from 302-279-3069 that I let go to voicemail. On the voicemail, there was a threatening but obviously scripted monologue along the lines of...
I'm calling regarding an enforcement action executed by the US Treasury requiring your serious attention. Ignoring this will be an intentional attempt to avoid initial appearance before a magistrate judge or Grand Jury for a federal criminal offense. My number is 302-279-3069. I repeat 302-279-3069. I advise you to cooperate with us and help us to help you. Thank you.
I especially like the "help us to help you" part.  This is basically the same phone scam reported by many others in tax seasons.

I encourage anyone with free time to give this number a call (dial *67 before you call them to mask your caller ID). If they're on the phone with you, then they won't be on the phone with some poor person that they've tricked.

## Monday, January 19, 2015

### A New Way to Download ePub NOOKBooks from Barnes and Noble

Back in September, Barnes and Noble removed the ability to download DRM'd ePubs directly from their site. If you are still using NOOK for PC or NOOK for Mac, you might be able to use them to download your newly purchased books and then find the ePubs in some obscure directory on your machine. However, that doesn't help Linux users. Moreover, those Desktop apps are not supported anymore, and so it is not an ideal solution.

So what do you do? If you have an Android device (in principle, you could use an Android emulator as well), you can download the NOOK e-reader Android app and, through a few steps, get the same ePubs you used to download. Here's how you do it:

4. Once the download is finished, open a file manager application like ES File Explorer and navigate to
/sdcard/Nook/content
You should find your NOOK book ePubs in that folder. That is, find the Nook folder in your internal storage, and then find the content folder inside it.
5. You can use your file manager to copy those ePubs out of that folder, or you can "Share" them to another installed app like Dropbox or Google Drive so that you can easily get them on other devices or your Desktop computer

[ Side note: You might try to plug your Android device into your computer and use MTP to access /sdcard/Nook/content directly from your Desktop and copy files out of it. However, it's possible that the ePubs won't be accessible this way. In fact, the might not even show up in the file listing. So you might be stuck using a file manager on your device. ]

## Tuesday, January 13, 2015

### Dollar Shave Club: My review and a cheaper alternative

I recently signed up for the Dollar Shave Club (DSC), which is a site that lets you paying $6 every 1 or 2 months for 4 razors by mail (and you can pause your subscription at any time). It turns out that you can get the exact same razors for much cheaper with no monthly subscription so long as you're willing to buy in bulk. Apparently DSC gets its razors from Dorco USA, which you can buy directly from on-line. In fact, you can buy retail handles with trial razors from Dorco too. For example, for$6, you can try out a Dorco handle and two razors, and then you can buy 36 replacement cartridges for less than $1/razor. I tried my "4X" DSC (or Dorco USA) razor for the first time this weekend. It was an OK razor. It had a nice heavy handle, but the angle of the pivoting head was not as steep as I'd like, and the moisturizing strip was far too thick. Consequently, it was very awkward to get into small nooks -- like under the nose. The "4X" razor doesn't have a trimming edge either (the more expensive 6-blade "Executive" model does), which would really improve matters. But after the shave, my face felt pretty good. It just took a lot longer to get through a full shave. So I'm not sure I can fully recommend these razors over, say, Schick Quattro Titanium disposable razors, which actually have a trimming edge and are pretty cheap from local stores or from Amazon (around$7/3 razors).

DSC also sent me a tester of their "Dr. Carver's Easy Shave Butter," which actually burned my face a bit as it went on (but I have sensitive skin). My face was fine afterward. But using Nivea For Men Active3 Body Wash for Body, Hair & Shave worked just as well without the sting. So I'm not sure I can recommend the Shave Butter either.

So I'm torn. It's possible that the 6-blade+edging "Executive" unit will be better. Either way, in the long run, it's cheaper to buy from Dorco USA directly, and I like the idea of dropping the subscription service. Dorco USA also sells other razor models and even disposable models. They also sell razors designed for women, whereas DSC just tells women to buy the 4X. The 4X has a handle which is somewhere between what you'd want for the face and the leg, but I think it's biased toward face shaving. So the lady's models at Dorco USA are definitely recommended if you need to shave legs.

So that's my brief review. tl;dr -- Check out DORCO if you like DSC's blades but don't like the subscription service.

References:

## Monday, December 29, 2014

### Looking for a cheap Bluetooth-capable portable MP3 player? Buy a no-contract smartphone and don't activate it!

My father needed a portable Bluetooth-capable MP3 player with large or expandable memory. Portable MP3 players with such features typically sell for $80+ (Trio mini), with the nicest units being at least$150 (iPod nano) or much more. The cheap models have terrible battery life (3 hours for Trio mini). Creative solution? He bought a no-contract LG Optimus Exceed 2 from Verizon Wireless for less than $50 at Best Buy and used a 16GB microSD card that he already owned.  LG Optimus Exceed 2, Prepaid (no-contract) Phone from Verizon Wireless This particular no-contract phone is fast, runs Android 4.4.2, and can be used without activating. In fact, it is very easy to bypass the initial activation screen with a special key sequence – volume up, volume down, back, home – pressed after restart at the language selection screen. After bypassing that screen just once, the phone is happy to run unactivated indefinitely after that – it will never present that activate screen again. Use airplane mode to prevent phone from looking for a network to save battery, and flip on WiFi and Bluetooth. It's like a$50 iPod touch. Nice deal.

You can get this phone from a number of places, as shown in links below. Of course, there are other no-contract phone choices you could make (check all no-contract carriers – Boost Mobile, Virgin Mobile USA, T-Mobile, etc.). If you're willing to compromise on the phone software or other capabilities, you can get decent "MP3 player phones" for $30 or even$20.
For example, you can get essentially the same phone from Boost Mobile packaged as the LG Realm for only $35 on Amazon right now. It's$40 from other retailers.
 LG Realm, Prepaid (no-contract) Phone from Boost Mobile
So that's like a \$35 iPod touch. Even nicer deal!

## Monday, February 04, 2013

### LaTeX template for NSF-style Biographical Sketch

On large multi-university NSF grant proposals, NSF requires that senior personnel submit a 2-page biographical sketch ("biosketch") that is formatting according to certain rules in their Grant Proposal Guide (GPG). The format is pretty simple, and so there does not seem to be much demand for a solid LaTeX template for one. Nevertheless, I thought some people might find one helpful. I've posted a PDF of my NSF-style biosketch below along with the TeX source used to generate it. On a related note, you can also find my curriculum vitae (CV) tailored for faculty searches (PDF, TeX) as well as for industry searches (PDF, TeX).

## Thursday, January 24, 2013

### Discrete-time Phase Portraits?

I was contacted recently by e-mail asking how to produce a phase portrait of a discrete-time system. In my initial response, I explained that a true "phase portrait" wasn't defined for discrete-time systems because the technical notion of a phase portrait depends on a special structure that comes along with ordinary differential equations. The original poster needed some additional clarification, and so I sent a second e-mail that I have posted below. It touches a little bit on the original poster's question, it comments on differences between discrete-time and continuous-time systems, it talks a bit about chaos, and it gives a brief description of Poincaré/return maps that are often used in the study of approximately periodic systems.

My point is that a discrete-time system really cannot be interpreted within a field-theoretic framework. A "phase portrait" captures both position and momentum of a continuous-time system described by an ordinary differential equation. These momentum variables setup the "field" that gives structure to the phase portrait. In a discrete-time system, we don't have the same kind of momentum.

For a continuous-time system, I can plot a point at an individual position, and I can also then draw a vector pointing away from that point representing the velocity at that instant of time. It is these velocity vectors that are put together to make a phase portrait of the system.

For a discrete-time system, there is no point-based velocity. In order for me to calculate the approximate "velocity" at a point, I need to know the position at the next point. Then I can draw a line between those two points and approximate the "velocity" of the system as going from the first point to the second point. However, if I have to know the next point in future anyway, it's more useful to just draw the second point.

Now, some discrete-time systems have a more predictable structure. For example, if you have a linear time-invariant discrete-time system like:

x[k+1] = M*x[k]

then the algebraic structure of the "M" matrix gives us insight into how trajectories will evolve. So for these special cases, it is possible to draw a kind of "phase portrait" for the discrete-time system. However, this is primarily because such a discrete-time system can be viewed as a sampled version of a continuous-time linear time-invariant system which does have a phase portrait.

So, for an arbitrary discrete-time system, the best thing you can do is explore trajectories from different initial conditions. Gradually, as you explore the space more and more, you may find boundaries of attractors (possibly strange attractors). A complication with discrete-time systems is that the "next" point may be very far from the "previous" point. Take, for example:

x[k+1] = -1.1*x[k]

If you start at x[0]=1, the trajectory will bounce from point clustered above 1 to points clustered below -1. In a continuous-time system, you might expect to see initial conditions above 1 stay near 1 and initial conditions below -1 stay near -1. That is, in a continuous-time system, you wouldn't imagine trajectories could cross x=0 (which is an equilibrium/fixed-point of this system). However, the discrete-time system can jump wildly from point to point.

Take, for example, the Henon map you mention. Wolfram's Mathworld has some nice plots:

http://mathworld.wolfram.com/HenonMap.html

The first pair of side-by-side plots are colored "according to the number of iterations required to escape". That is, the plots were generated by starting at several initial conditions and recording the resulting trajectories. Each "iteration" gives the next point from the previous point. For a while, a trajectory will stay around its initial condition. Eventually, it will escape and move away from the region. The regions are colored based on how many iterations (i.e., how many calculations after the initial condition) it took for the trajectory to leave the region.

The second pair of side-by-side plots show a SINGLE trajectory started at x[0]=0 and y[0]=0. Each point was recorded and gradually a pattern emerged. Notice how in the left plot the two regions appear to be disconnected. If you saw a phase portrait that looked like this in a continuous-time system, you would conclude that initial conditions within one region would not be able to join the other region for this set of parameters. However, this plot was generated from a SINGLE initial condition. So the plot jumps from points in the top left to points in the bottom right and back.

So that's how you can explore something like the "phase space" for discrete-time systems. You can probe it with different initial conditions. For chaotic systems, you have to be very careful you don't accidentally jump over an interesting region of initial conditions that may have qualitatively different trajectories that follow from them.

As an aside, I guess it's also worth mentioning that many popular discrete-time chaotic maps are actually Poincaré maps of continuous-time dynamical systems. Poincaré maps have other names, including "return maps." Consider, for example, the planets as they orbit the sun. The actual orbits of the planets in three dimensions looks like a tangled mess when you consider their histories over several cycles around the sun because each orbit is slightly different than the previous orbit (i.e., they aren't entirely planar). However, if you insert a plane perpendicular to their orbits at a single location, each planet pierces the plane at one point every cycle. The resulting shapes that are poked out of that cross section reveal structure in the orbits.

I hope that helps! --
Ted

## Tuesday, October 09, 2012

### natbib-like frontend for chicago-style macros

For some reason, the ACM has baked in chicago-like citation macros into the ACM SIG proceedings LaTeX templates instead of using the far superior natbib that literally everyone else on the planet uses. I'm much more accustomed to typing \citet as opposed to \shortciteN, \citeauthor as opposed to \shortciteANP, etc. So I decided to add this little translation table to my preamble:
\def\citet{\shortciteN}
\def\Citet{\shortciteN}
\def\citeauthor{\shortciteANP}
\def\Citeauthor{\shortciteANP}

\let\chicagociteyear\citeyear
\def\citeyearpar{\chicagociteyear}
\def\citeyear{\citeyearNP}

\def\citep{\shortcite}
Unfortunately, that means I can't use any of the optional arguments to \citep or \citet. If you are just looking for a translation table so you can use the chicago-style macros directly, try this:
natbibchicago
\citet or \Citet\shortciteN
\citeauthor or \Citeauthor\shortciteANP
\citeyear\citeyearNP
\citeyearpar\citeyear
\citep\shortcite
It's crazy how chicago-style macros sometimes use "N" to mean "noun" and other times to be the first half of "NP" being "no parentheses."

## Thursday, August 09, 2012

### CTAN search engine definition for Firefox

If you're looking for search engine definitions for CTAN that are compatible with Firefox's search toolbar, then visit:While on that page, click on the down arrow in the icon in your your Firefox search toolbar. You should see several "Add" options at the bottom. That lets you add several different types of CTAN search to your toolbar. You can then "Manage Search Engines" and add a keyword to some of them. I use the keyword "ctan" to search package ID's and the keyword "ctanp" to search over package ID's and descriptions. The keyword lets me search directly from the awesome bar without having to use the search bar. [ These definitions were inspired by firefox_ctan_plugins by Martin Engler, but those don't work with CTAN anymore. ]

## Sunday, July 29, 2012

### Bookmarklets for Arizona State University (ASU) library proxy

This message goes out to ASU faculty, staff, and students. Doesn't it make you mad when you're off campus and you want to read a paper and the research database that holds the paper doesn't let you view it because you're not a subscriber? I hate that.

Luckily, the ASU library system provides an off-campus scholarly portal from my.asu.edu (under the "Library" option on the left) that can be used to make your off-campus connection look like an on-campus one so research databases will grant you ASU-caliber access. Unfortunately, if you've clicked on a link to a paper, you have to re-do your search within their portal to get your off-campus access, and that's really inconvenient. So I've come up with a shortcut (similar to my OSU LOCSI shortcut).

Drag the bookmarklet below to your "bookmarks toolbar" (that strip of bookmarks that rides just below the "location bar" in your browser).
• ASU LOCSI ("LOCSI" = "Library Off-Campus Sign-In")
When you're on a page you'd like special access to (e.g., the official site of an academic paper of interest to you), click on the bookmarklet and you'll be transported to that site via ASU's off-campus sign in. You may have to use your ASU username and password to login the first time you click on the bookmarklet, but after that you should be granted ASU-level access quickly.

(feel free to rename that bookmarklet as you wish; the name "ASU LOCSI" isn't important to its function)

I hope that helps.