Electricity is created by using a variety of methods but most electricity is generated using a fossil fuels and these fossil fuels are very limited and will run out on us eventually.
When we use more electricity then the electric generators have to work harder to produce the electricity that is required.
And that uses more fossil fuels. But just having a few lights on in your house is not really make a difference in how much fossil fuels are being used since the electricity is being generated anyway.
The problem is when electric usage goes up really fast such as electric heaters and central ac or other huge energy hogs.
Then they have to run more generators to keep up with the electric demand and that uses more fossil fuels.
But now that we have renewable energy forms such as solar and wind generation I wouldn't worry as much about wasting too much electricity.
Heck I don't really even care if we eventually run out of fossil fuels because I will be dead one day and running out of fossil fuels won't affect me.