The explanation given for this is that the faster I go, the more I slow down through time, so from my reference frame, light decelerates (or accelerates? I'm not sure, but it actually doesn't matter for my question, so if I'm wrong, just switch them around mentally as you read).

Perhaps I'm reading this wrong, but it seems you're assuming that time slowing down is an absolute, not a relative, effect. Do you think there is an absolute fact of the matter about how fast you're moving? If you do, then this is a big mistake. You only have a velocity relative to some reference frame.

If you don't think of velocity as absolute, what do you mean by statements like this one:

The same would apply to light, but because time has slowed for me, so has the light from my perspective.

There is no absolute fact of the matter about whether time has slowed for you. This is only true from certain perspectives. Crucially, it is not true from your own perspective. From your perspective, time always moves faster for you than it does for someone moving relative to you.

I really encourage you to read the first few chapters of this: http://www.pitt.edu/~jdnorton/teaching/HPS_0410/chapters/index.html

It is simply written and should clear up some of your confusions.