The reason the senate looks like it does, in particular with regards to the wildly uneven representation, probably has less to do with the "tyranny of the majority" issue than it does with other factors (however, a whole lot else about the senate is. Like its existence and consideration as the "more mature" legislative body, and the fact that senators were not originally directly elected).
The reason the senate is the way it is is because before The Constitution, there was The Articles of Confederation. Under these laws, the United States had a very weak central government that was barely effective at all, and couldn't really enforce laws. The states were reluctant to give up any powers to a federal government, but then they were convinced. By, among other things, the poor people revolting, and a total inability to deal with this.
Anyways, there was a debate over the legislature of the new government. Under the Articles of Confederation, since the barely-existent federal government was a confederation of states, each state was proportionally represented. The Constitution would be creating a governing body that looked a lot closer to an actual nation, and a number of states proposed a proportional representation model. The Virginia plan called for a bicameral legislature, with the "lower" house having its members directly elected, and its "upper" house having its members chosen by state legislatures. Which was how our house and senate worked until senators became directly elected later on, but with a proportional senate. Meanwhile, smaller states tended to prefer a system where each state got the same number of representatives (for reasons that should be obvious: these smaller states still wanted to enjoy the equality of power and influence that they had had up to now). The New Jersey plan suggested a Unicameral legislature with equal representation for each state and a weaker federal government overall.
Nobody was yielding, and so there was a compromise: the Virginia plan would proceed, but with equal representation for each state in the senate. In the house, representation would be proportional. But this is also where the three-fifths compromise comes from... and why it's called that.
So that's why the Senate is wildly uneven in how many constituents it represents: a struggle for power between the states, a fear of the dominance of large states, and a bit of horrible horrible racism thrown in for good measure.