Apparently, up until the middle ages, the custom of reading the entire Torah in the course of a year was not firmly established throughout the Jewish world. There are collections of sermons in the ancient rabbinic literature that are based on a sequence of over 150 Torah readings; in other words, the Torah was divided into smaller portions, read weekly over three years. This was the custom in Eretz Yisrael. In Babylonia, the procedure that developed was to divide the Torah into 52 portions and complete the whole reading in one year. We can see from the Rambams comment in the 12th century that the Babylonian custom was almost though not quite universal by then:
The prevalent custom among all Israel is to complete the reading of the Torah in one year. It is begun on the Sabbath after Sukkot, when the portion beginning with In the beginning is read and so on until the entire Torah is completed on Sukkot. Some complete the reading of the Torah in three years, but this is not the prevalent custom.
-Mishneh Torah, laws of prayer, 13:1
What we call Simchat Torah is not mentioned in the Bible or the Talmud, and seems to have originated at around the time that the yearly cycle of readings became universal. The Bible does command, of course, the observance of the day after the end of Sukkot as a holiday, but with no particular behaviors other than abstaining from work, and bringing certain sacrifices (Leviticus 23:33-36). This day, Shemini Atzeret, is kind of an orphan holiday attached to Sukkot, but without the lulav, etrog, or sukkah. In the Diaspora, Shemini Atzeret, like the first day of Sukkot, was observed for two days, due to the tradition of uncertainty about its exact date. And if Shemini Atzeret was an orphan how much the more so was the second day of Shemini Atzeret. And so, sometime around 1000 C.E., the second day of Shemini Atzeret got the added meaning of a celebration of the re-starting of the Torah cycle, which occurred on the following Shabbat. Some poems were added to the liturgy, and a procession with the Torah scrolls, and the custom of reading both the end of Deuteronomy and the beginning of Genesis. Thus, not only is the annual re-starting of the Torah reading a Diaspora custom, but the holiday celebrating it has a strong Diaspora nature as well occurring on a day that in Eretz Yisrael is of no halachic significance (second days of festivals are not observed in Israel).
Over the years various customs were added to emphasize our love for the Torah and the importance of reading it. For example, the custom of reading the portion over and over again until every member of the congregation has had a chance to be called up for an aliyah; or of a special aliyah for all the children; or the Hasidic custom of dancing with the Torah scrolls with great fervor long into the night. Most of us have probably already forgotten that in the 1960s, Simchat Torah became a remarkable public celebration of Jewish identity among the Jews of the Soviet Union, a powerful symbol of their longing for and pride in the Jewish tradition.
In Israel today, Simchat Torah is observed on the one and only day of Shemini Atzeret. Hasidic-style dancing is widespread, at all kinds of synagogues. However, for most people it is just another day of Sukkot vacation. In many towns, the night after Simchat Torah public celebrations are held simulations of Simchat Torah dancing and singing, but with professional musicians and electronic amplification. These are called hakafot shniyot (second processions) in other words, reruns of Simchat Torah, designed to allow the public to enjoy the holiday without any of the restrictions or accoutrements of religion. These public festivals are usually popular and crowded. This days odyssey has been remarkable from Diaspora religious observance to Israeli secular festival. It is a classic case study in the re-valuation of Jewish tradition in the Jewish state. It will be interesting to check on its status in another few generations.