Reasoning about it from the perspective of something physical, one could argue that intellectual property doesn't exist at all, and we should only consider the books, movie media etc. to be property.
If we were to take the contents of a book or a movie for example, and copy it, you still have the physical source, and nothing is lost, your property is still yours. It's just that there is more of it due to the additional copy. That copy isn't yours, and from the moment it was created it still isn't yours. So in that line of thinking, the property that was created is not the same as the property it was copied from, which means two different properties exist.
We can make this even trickier, because if we were to reason about the physical property and the intellectual property separately, the story in a book, and the physical book itself would be two different properties. So when you create a copy, that book that started out blank was definitely not part of the property of someone else. So does the act of adding intellectual property now suddenly transfer the physical property to the source of the intellectual property?
In the current laws and practises around the world we have made all sorts of rules about this, but just reasoning about it before falling back on established practice already shows that it doesn't always turn out to be as easy as it seems.
"Hey, I like that table, I'm going to commission somebody to build one for myself!"
Totally legal. No loss to first individual. The fact is that intellectual property IS treated differently, and was justified as being required to incentivize invention and creation NOT because there was any natural right to the product of one's thoughts to not be copied or expanded upon. Now that it arguably gets in the way of innovation and creation, what justification is there for these ADDITIONAL legal restrictions.
>NOT because there was any natural right to the product of one's thoughts to not be copied or expanded upon
grandma not sharing her best recipes, and children calling each other "copycat!" is an argument for a natural right. It is innately how humans feel, and there is a supporting argument that we don't want grandma's secret innovations dying with her, to incent sharing.
> While I agree with you, I'm struggling to find a good argument why intellectual property should be treated differently from physical property
1) First reason, it is not physical property.
2) Second, many of the creators who lobbied for longer copyright terms benefited from a rich public domain when they did not have anything. In the case of Walt Disney, he made a film series based on Alice in Wonderland, which never had copyright protection in the US, and it's copyright expired in the UK in 1907.
Disney also used "Le Sacre du printemps" in "Fantasia" after meeting Stravinsky, who was offended by the idea. However, Disney informed him that he would use it anyway since it was not protected in the US. Stravinsky was paid pennies.