PFD - Make sure you have & are wearing a good PFD. There are a lot of different typea available. Personally I recommend the NRS Chinook. It's specifically made for kayak fisherman with comfort & mobility in mind.
Whistle - Have a good "pea-less" whistle. Have the whistle attached to your PFD so that it will not be lost.
Head Lamp (Night Fishing) - This is a necessity if you plan on fishing after dusk. Not only does it help with you being able to see. It can be used for signialing if you are in trouble.
VHF Radio - While it is recommeded you not fish alone, it is not always possible. If you are going to fish alone first tell someone where you will be while out. Second, invest in a VHF radio. There are quite a few to pick from just make sure it is waterproof. I recommend chosing one small enough to attach to your PFD so that you have it in the event you "turtle" your kayak.
Flags & Lighting - Remember you are almost always the smallest craft on the water. You need to have a pole topped with a light & flag to make you & your kayak more visible on the water. You can purchase something light the VISIcarbon Pro or you can chose the DIY route.
Self Rescue - Self rescue means righting & re-entering your kayak without assistance. I recommend practice self rescue with a friend. Self rescue it difficult on calm water (which I have had to do once). I can imagine the difficulty if you were in choppy seas. At the very least watch this video by Len Zilkowsky.
Everything listed above are necessities. As with any outdoor activity make sure you are dressed appropiately for the the weather. Have suncrscreen handy if it is summer. You will burn very quickly between the sun & reflection from the water. Regardless of season make sure you stay properly hydrated.