Generally I have avoided siding out any of the "core" parts of the affinity deck. Affinity is too linear to remove anything like Frogmite, Myr Enforcer, or Thoughtcast. Removing any of your fixing (Chromatic Star, Springleaf Drum, Prophetic Prism) is also essentially impossible. So that leaves you with the following options to take out:

Galvanic Blast
Gearseeker Serpent
Ichor Wellspring
Perilous Research
Atog / Fling combo parts
There are tons of pauper matchups so it's hard to give you an exact guide for every match, but here is what I generally do.

Mirror

The mirror is usually decided pre-board by who is more explosive, and if it's about equal, by who sets up the Flingatog combo first. Post-board, I like to take out some of the slower draw for Gorilla Shaman (who can win on the spot), and Hydroblast. If you Hydroblast their Fling or post-Temur Atog, you just win.

Out: 2x Ichor Wellspring, 2x Perilous Research, 1x Atog
In: 3x Gorilla Shaman, 2x Hydroblast
Creature-based aggro (Stompy, Goblins)

Against these decks you are usually not the beatdown. Side in Krark-clan Shaman to clear the board. I've found that Affinity is very favored against these decks so it doesn't need much help.

Out: 1x Gearseeker, 1x Ichor Wellspring
In: 2x Krark-clan Shaman
Bogles

Pre-board, Bogles is non-interactive. Whoever is faster wins. Post-board, Affinity's sideboard cards are basically I-Win buttons.

Out: 4x Galvanic Blast, 1x Gearseeker Serpent, 1x Atog
In: 2x Krark-clan Shaman, 2x Electrickery, 2x Tranquility / Serene Heart
Elves

Again Affinity doesn't need much help against Elves. Keep Galvanics to hit their actual threats (Timberwatch, mostly). Don't be afraid to Electrickery on two elves only, especially if they only have one forest.

Out: 2x Ichor Wellspring, 1x Perilous Research, 1x Gearseeker
In: 2x Krark-clan Shaman, 2x Electrickery
Burn

Like Bogles, this matchup is pretty non-interactive. I have one Feed the Clan for Burn in my sideboard, but it's not a terribly frequent matchup so you could consider not even running that.

Out: Gearseeker Serpent [Edit: And 1x Perilous, 1x Wellspring]
In: Feed the Clan [Edit: And 2x Hydroblast]
Skred / Delver Midrange

Electrickery can blow them out if they are on a Fae plan. Keep a couple Galvanic to hit Ninja or flipped Delver. I don't side in Hydroblast because I tend not to care what they hit with Skred. They can only win by getting a lot better than one-for-one. Save your Pyroblast to protect Flingatog if you can.

Out: 2x Galvanic Blast, 1x Gearseeker, 1x Perilous, 1x Ichor Wellspring
In: 3x Pyroblast, 2x Electrickery
Dimir Control / Tron / Teachings

These are the tough matchups, but I haven't figured out if there is a great sideboard strategy to improve them. Generally you win by being more explosive than their early game can handle. To that end it's important to toss back do-nothing hands (i.e., turn 1 nothing, turn 2 prophetic prism). I ran Relic of Progenitus for some graveyard-based matchups for a while, but it always felt so pointless. If anyone has sideboard suggestions, I'd love to hear them. Resourceful Return? Reaping the Graves??

Generally:

Out: 3x Galvanic Blast
In: 3x Pyroblast

Boros Monarch/Kuldota

They are pretty slow and non-threatening, but can make it difficult to end the game. If you're seeing a ton of Kuldotha decks and you want to tune your affinity to it, I would swap a fling in the maindeck for a Temur Battle Rage, which make your Atogs lethal through their chump blockers earlier. As for sideboard:

Out: 2x Flayer Husk, [Optional: 1x Wellspring, 1x Perilous]
In: 2x Electrickery, [Optional: 2x Gorilla Shaman]
Shaman can be good if you catch Furnace or Den with it, and can blow up clues. In the late game it can stop their card draw recursion by killing Prism. I'm not sure it's worth it though. You want to push through early damage and finish them off with Galvanic or Flingatog, and the wellspring/research combo digs you to the combo. I'd rather side out Husk than the Gearseeker; the big body is relevant and can occasionally be unblockable in a longer game, and the early explosion Husk offers is not quite as needed against their slower gameplan.

Be aware of whether they have Prismatic Strands, which is usually their only defense against your finisher (if they are playing COP:R, you're probably winning anyways)