That's part of the dilemma. And I'd say it is the limited thinking we are forced to be slaves of. The brain is still evolving. What you describe is linear thinking. There must be a start, there must be an end...that's the limit of the comprehension, however I can give you an example of infinity.
Take mathematics. You draw a line and make a point on that line, you define it by giving it a number. You can divide that number further down, take this number and divide it further down... you get the picture. You can keep doing this into infinity. There is no pot of gold at the end of the rainbow. So take that analogy and look at the universe. It never started to be, it was always there and it will never end to be.
Yes, that is mind-blowing but it is logical.