Skip to content
April 9, 2014 / like2think

Annealing rate

When using gradient descent it is comfortable to have the learning rate decreasing exponentially, that is \epsilon_n = \theta^{n} \epsilon_0. Setting \theta is then important. Suppose I have chosen desirable \epsilon_0 and \epsilon_n after n steps. How to find the corresponding \theta without a calculator?

First of all usually \log_{10} \epsilon_0 and \log_{10} \epsilon_n are easy to calculate (cause they are frequently just powers of ten, simple, isn’t it?). Then \log 10 = 2.3 – easy to remember. Finally \theta is usually close to one, which means that \log \theta \approx \theta - 1. Here is an equation that binds all these guys:

n = \frac{-\log_{10} \epsilon_n + \log_{10} \epsilon_0}{-2.3 \log_{10} (1 - \theta)}

It is even simpler to use it in the opposite direction: just remember that your learning rate becomes ten times smaller each 2.3 \cdot 10^{-\log_{10} (1 - \theta)} iterations.

February 23, 2014 / like2think

The word of the day 4

I really missed this one: to let sb down. Having it heard in a song of Bob Marley I did not know the meaning:

to disappoint someone by failing to do what you agreed to do or were expected to do

…brought you by Cambridge dictionary. Now I have a good translations for Russian подводить, hooray!



February 16, 2014 / like2think

The word of the day 3

Here come three words with close meanings (at least for my taste): smug, arrogant, conceited. Let’s look at definitions from Cambridge dictionary:

smug – too pleased or satisfied about something you have achieved or something you know

arrogant – unpleasantly proud and behaving as if you are more important than, or know more than, other people

conceited – too proud of yourself and your actions and abilities

I think arrogant stands out being more description of person’s behaviour, not of his self-esteem. I also feel that one can be smug only for reason (some achievement), and can be conceited just because he evaluates himself not fairly.

P.S. A discussion on this topic:


January 29, 2014 / like2think

Holes in my Machine Learning knowledge.

  1. ICA – I almost understood what it is but… still a hole.
  2. Kalman Filter – everybody know what it is except me.
  3. Decision Trees – however it seems that no deep theory exist in this area at all.
  4. Boosting  – the idea is potentially so powerful… yum-yum.
  5. Graphical Models – finally, this term!
  6. Collaborating Filtering – as a remarkable application area for automatic learning techniques.
January 19, 2014 / like2think

Mini-projects to do

  1. I want to learn to use vim as TeX editor. So far I have used Texmaker for this purpose. Eventually I realized that I don’t use most of its features. At the same time I miss advanced editing facilities of vim very much (and also my carefully selected plugins). The only reason why I still use Texmaker is its ability to jump to the place in pdf corresponding to given position in tex file and vice versa. But this feature is implemented using standard interface called synctex. A number of plugins for vim partially know to how to work with this guy, and what I actually have to do is find the right one. Hopefully some of them will do the job.
  2. I want to master python packaging tool aka distutils. The idea is that it would be nice to reuse own code without copying files and hacking with PYTHONPATH. Allow other people to try it fast is another benefit which I’m going to earn this way.
January 14, 2014 / like2think

The word of the day 2

The word which I have some original right to hate being a colorblind: tinge – a very slight amount of colour or feeling (stolen from Cambridge dictionary), used so frequently in books to refine appearance descriptions.


January 2, 2014 / like2think


Being a recent expat from a country with poor average command of English language I find it challenging to read novels with lots of fancy words in original. I’ve started compiling a list of internet resources that can be of help in such situation:

  1. The English-to-English dictionaries of renowned English universities (Oxford, Cambridge) are surely the most authoritative source of word explanations. All the articles there contains example sentences, for polysemantic words one will find several meanings listed, phrasal verbs are included. Additionally one escapes paying the price of translating into other language and enriches his vocabulary working with them. I would rate them as the number one tool in my toolbox, but even such a tool is not a holy grail. Sometimes it is almost impossible to make it without any translation to your native language and one example is often not enough. Finally it is not comprehensive since languages always evolve faster than official dictionaries.
  2. Google Translate – I think everybody knows this one. The big G offers you statistics based machine translation. This implies pros and cons: it’s versatile, sometimes capable to translate idioms, copes with inflected words. But reliability of automatic methods is limited, and what’s more important it doesn’t give you contexts for translation variants and any grammatical information. I never trust it fully and use it only as auxiliary tool: it can give you a valuable cue what to do next and also is a good spell checker :)
  3. looks like a perspective mixture of hand-crafted and automated approaches. It greets your with a verbose explanation of the word (I wonder how they had them all written… ) and offers a range of synonyms and examples. I can’t just use it and forget about all others primarily because I don’t know how much one can trust it. There is of course a lot of other tools like the, but so far this one seems to be fairly good to forget about others.
  4. Good hand-crafted bilingual dictionary can help you sometimes when the forementioned approaches fail. For me, the Russian language speaker, such a dictionary is Lingvo.
  5. Urban dictionary is irreplaceable if you have to deal with slang and/or obscene words.
  6. Translation through wikipedia works like this: you find an article of your interest in English, let it be, and search for your native language in the panel which  lists corresponding articles. For example this link informs you that curvature in Russian is кривизна. This method is especially useful for scientific terms.

Usually I go in top-bottom direction through such list when I need to understand what some involved word like “stunt” or “mockery”  means, skipping those options that obviously don’t suit to the particular case I work with.

Enjoy reading originals!


Get every new post delivered to your Inbox.