How to learn?

Grading rules

as on the main page of the course.

What programming languages will be using?

Example problem: "guess the number" puzzle

I have a secret integer number x from 0 to 1000. You can ask questions of form, "does x≤...". How can you find the value of x?

The idea of the solution: first, ask whether x≤500. After we receive the answer, we either know that 0≤x≤500, or that 501≤x≤1000. We ask about the middle number of the new interval, again splitting the interval in half. After about 10 questions we know the value of x exactly.

This algorithm can be described by the following fragment of a C++ program: 
int l = 0;
int r = 1000;
// l and r denote our current bounds: we know that l <= x <= r
while(l<r) {
  // we will ask about the number in the middle
  int m = (l+r)/2;
  // note: this is an integer division, thus e.g., for l=0, r=1 we get m=1/2=0
  if(x <= m)
    r = m;
  else
    l = m+1;
  }
After this fragment executes, both the variables l and r will be equal to x.
Although the idea of the algorithm is very easy, implementing it precisely enough so that it can be executed by the computer is not that straightforward. For example, if one writes l = x instead of l = x+1, the algorithm will not be working correctly: suppose that, at some point, l=0, r=1, and x=1. We compute m=0, and since it is not true x <= m, we set l to m, which does not change anything -- thus, we will keep asking about x <= 0 forever!

How to check that the program is correct then?

Application

The last problem has been shown as a puzzle, to show that algorithmics is not necessarily about computers -- it shows that algorithms are useful in general, whether using a computer or not. However, the algorithm shown, known as binary search, has lots of applications in programming.

For example, suppose we have an array of integers, i.e., a[0], a[1], ..., a[n-1], and we know that it is non-decreasing. Given v, we want to find the smallest x such that a[x] ≥ v; in case if all the integers in our array are smaller than v, we should return n. How to do this?

We use the same general idea as in the puzzle, but now instead of checking whether x <= m, we check whether a[m] >= v. We also start with r=n instead of 1000. We prove the correctness of our program in exactly the same way, except that the invariant says now that a[i]<v for each valid i smaller than l, and a[i]>=v for each valid i greater or equal to r.

The complete program

Below is an example of a complete C++ program which uses this: 
#include <iostream>
#include <vector>

// We separate our binary search as a function. For the array a, we use the type std::vector<int> from
// the standard library. 

int bin_search(const std::vector<int>& a, int v) {

  // The & sign denotes that 'a' is a reference, i.e., bin_search has an access to a std::vector<int> which
  // is declared everywhere. If & is omitted, the function makes its own copy of the vector given to it,
  // which takes lots of time -- thus it is important not to forget the & sign.

  // 'const' signifies that the function will not change the value of a. 
  
  int l = 0;
  int r = a.size();
  while(l<r) {
    int m = (l+r)/2;
    if(a[m] >= v)
      r = m;
    else
      l = m+1;
    }
  return r;
  }


int main() {
  // we construct an example array a:
  std::vector<int> a;
  for(int i=0; i<1000; i++) 
    a.push_back(i*i);
  
  while(true) {
    int v;
    std::cin >> v;
    std::cout << "The answer is: " << bin_search(a,v) << "\n";
    }

  return 0;
  }
And a Python version: 
def bin_search(a, v):
  l = 0
  r = len(a)
  while l<r:
    m = (l+r)//2
    if a[m] >= v:
      r = m
    else:
      l = m+1
  return r

a = [i*i for i in range(100)]

while 1:
  v = int(input())
  print("The answer is: "+str(bin_search(a,v)))
The important differences between the C++ and Python versions:

The importance of using efficient algorithms

A simpler algorithm for the last problem is the linear search, in which you simply look for the value of x from beginning to end: 
int x = 0;
while(x < n && a[x] < v) x++;
How important is using the faster algorithm, compared to using a faster computer, or a faster programming language? Consider the case of n=1000000000. In the '80s, in BASIC, the interpreter could do about 1000 simple operations per second. Therefore, the binary search (30 iterations) would take about 50 ms, while the linear search would take about 12 days. On the other hand, C++ on a modern computer can do about 1000000000 operations per second -- thus, the linear search will take about 1 second, while the binary search takes about 50 nanoseconds.

As this example shows -- using a faster algorithm is more important than using a faster computer or programming language. An efficient algorithm will be faster than a bad one, even when given a significant technological disadvantage! And even if answering one query per second does not sound that bad, we would usually use the binary search as a part of a bigger system, which needs to find a value in a array many times. Coders with no knowledge of algorithmics would just implement the linear search (thinking it should be fast enough), find out that their program works slowly, and then get amazed when told about binary search (even though it is a very simple algorithm).

The number of iterations is the binary logarithm of n for binary search, while n itself for linear search. To get the actual running time, we have to multiply this by a constant, which depends on the technology used, and how many elementary operations we do in each loop -- for example, if we considered the case of x=m separately, the number of operations per iteration would be different. While we could save time by changing this, or by improving our technology, the difference between log(n) and n is the most significant -- therefore, in our course, we will be the most concerned about making the order as small as possible. This will be explained in more detail in the further lectures.