Fabric Softener only softens the clothing or fabric and does not actually clean or wash the clothing or remove any dirt or grime from the clothing.
So washing clothes with fabric softener will only soften the clothes and not get the clothes clean.
So always wash the clothes with regular laundry detergent to get the clothing clean and then use fabric softener if you want to keep your clothing softened and feeling better.
So it won't harm your clothes to wash them with fabric softener only but your clothes will not get clean at all without the use of laundry detergent because fabric softener is not a laundry detergent or cleaning solution.