"The third millennium" refers to the third period of one thousand years since the start of the AD era. The first day of this era was 1 January AD 1. Due to the lack of mathematical sophistication of the original deviser of the calendar, there is no year zero: the year 1 BC is followed by the year AD 1. Simply because there is no year zero, one thousand years after the start of the AD era is 1 January 1001, and two thousand years after is 1 January 2001.
Of course, it's not actually as simple as that. See, for example, Peter Meyer's When Did the New Millennium Begin?. However, given the entire basis of the numbering scheme, AD or CE, is the (estimated) birth of Jesus Christ, it seems to me to make more sense to measure 2000 years from that point.
-- Hugo Tyson, .sig (on cam.misc)
Given the very common misspelling of millennium, there have been suggestions that the millenium starts on 1 January 2000, whilst the third millennium starts on 1 January 2001.
Given the discussion above, and the nature of the bug itself, the so-called millennium bug is more properly called the Y2K (year 2000) bug. It is a computer bug caused by "efficiently" storing year dates using only two digits, so that 80 is interpreted to be 1980, 99 is interpreted to be 1999, and 00 is interpreted to be ... what, precisely?
Another computer bug for the year 2000 is a leap year bug, because not all computers recognise the existence of 29 February 2000.