What would need to be done in order to load .ics files with fullcalendar? I cannot use php or .net unfortunately.
what you would need is to write your own extension to fullcalendar (similar to the gcal.js which is provided with fullcalendar) something you could call ical.js
You should know that writting an complete ical parser can be quite draining so you may want to consider sticking with google calendar for your back-end unless you have a compeling reason.
Should you go down the road of developping your own extension for fullcalendar you may want to have a look at an existing jquery ical parser (here - disclaimer: I've never tried this plugin)