Can we walk away from buyer’s agent?

We’ve been touring houses for a year now but haven’t found anything in our budget. We’ve been working with a buyer’s agent who sometimes comes out on the tours with us and get us documents.

Now a friend of a friend is selling their house and offering us an off-market sale. It’s really tempting. Just talked to the seller’s agent who said “the expectation is he’d represent both parties.”

Can we just… walk away from our buyer’s agent to do the dual representation plan? We haven’t signed any kind of contract with him…

This is in Seattle, in case that matters. I guess Northwest MLS is the local MLS.

P.s. anticipating questions: the house is in good shape. The price is exactly our budget. But I think it’ll sell for more than they’re asking if we wait till it’s publicly listed.