Short answer
Yes, a small cat eating a whole kumquat is riskier. The peel's oils are more concentrated for their size and can cause vomiting, drooling, or stomach upset. Remove any remaining fruit and watch for these signs over the next few hours. If your cat starts drooling or vomiting, call your vet for advice.
Kumquat — Food safety facts
Feed with care · Cats · full breakdown on the linked food page
Citrus peel oils can cause vomiting and drooling. At most a tiny piece of flesh.
The full verdict, portion guidance and serving notes are collected on the Kumquat food facts page.