Every retailer is well aware of the challenges that accepting credit cards can bring. It extends far beyond the security risks that are on the table like keeping card information safe once it is processed or the fallout of accepting a fraudulent one. However, there is also the more rudimentary issue of the price tag that comes with just accepting this form of payment.

All credit card processors charge a small fee — the average is 2 percent — to handle every transaction that is handled with plastic. Up until October 2011, banks were also able take up to 45 cents per swipe, that number has since been capped at 21 cents. While this seems like a small number it can add up quickly and these fees have tripled over the previous decade and total $50 billion a year.

In a recent article from the National Retail Federation, the trade association examined the latest news surrounding swipe fees.

There have been two major cases over the last year. In July, U.S. District Court Judge Richard Leon ruled that the 21-cent cap for banks was too high because it was not in proportion with the level ordered by Congress. The judge ordered the Federal Reserve to re-crunch the numbers and come up with a lower level. This is currently be appealed by the Fed.

In December, U.S. District Court Judge John Gleeson ruled on a lawsuit between a group of retailers and card providers Visa and MasterCard that resulted in a $30 billion settlement. This has since turned into a quagmire of additional legal issues. Some retailers felt they would not be getting enough when the money was handed out and others that had filed suit did not agree to the settlement in the first place. Since then, several retailers have pulled out of the settlement in order to file their own lawsuits. All of these are still pending or are in appeal.

This shows that something as simple as a credit card swipe fee of 2 percent can have a billion dollar impact on the industry. And don't think that this issue is only between card providers, banks and retailers. Because retailers have to pay more, that price is often passed along to the consumer.

"Many retailers have cited swipe fees as their second or third highest cost behind salaries and employee health benefits. With retail industry profits averaging only about 2 percent, there is no room for retailers to absorb the expense, so swipe fees are passed on to customers in the form of higher prices," the article reads. "In addition, card industry contracts and practices have required that merchandise be priced at the credit card price—including the swipe fees—and have made it difficult to either show the fees to customers or to offer a cash discount."

The NRF estimates that swipe fees cost the average U.S. household roughly $400 a year. This also hurts retailers because higher prices means consumers are only buying the essentials, meaning less sales overall.

The swipe fee is one of the biggest challenges in the business landscape and retailers are trying to find ways to make this right. Having the right credit card payment processor is a strong step but there will be changes coming as these legal issues are settled one way or another.