There's no need to throw away the bits of fish that you cut off when you filet them and so forth. Fish bones and heads can be kept for making soup. Or, if the bones are tender enough they can be made into delicious fish-bone crackers.
At the sushi restaurant in New York I worked at many years ago, the chefs used to serve these as extra treats to favored customers. One of those was a lovely little girl, who used to come regularly with her father. She just loved those fish bone crackers. So, one year the chefs made a big batch of them and gave her a takeout box full for her birthday. She was so happy I thought her eyes were going to pop out of her head.
I've paired these with shoestring potatoes, which taste surprisingly sweet next to the umami-rich fish bones. The type of potato is important - choose a nice firm waxy type, not a floury type like Idaho baking potatoes. Alternatively you can use sweet potatoes.