Skip to content

Conversation

Reskov
Copy link

@Reskov Reskov commented Jun 21, 2021

  • removed double iteration over self.columns at the _do_load method
  • prepare the property to column mapping only when the column value was changed
  • removed extra loop over row values to check if all values are none

@Reskov Reskov force-pushed the optimize-do-load branch from 501f344 to e766dd3 Compare June 22, 2021 02:20
@Reskov
Copy link
Author

Reskov commented Jun 22, 2021

It is still pretty slow, but results are better on my laptop

@contextmanager
def timer():
    real_time = time.time()
    cpu_time = time.process_time()
    yield
    real_time = time.time() - real_time
    cpu_time = time.process_time() - cpu_time

    print(f"CPU time: {cpu_time:.2f} s, Real time: {real_time:.2f} s")

async def test_model_alternative(user, event_loop):
    row = await User.query.gino.load(lambda r, c: r).first()

    # async def new_load():
    loader = ModelLoader(User)
    ctx = {}

    with timer():
        print("Full model")
        timeit.timeit(lambda: loader.do_load(row, ctx), number=20000)

    loader = ModelLoader(User, User.nickname)
    with timer():
        print("Single column loader")
        timeit.timeit(lambda: loader.do_load(row, ctx), number=20000)

    row = await db.select([User.nickname]).gino.load(lambda r, c: r).first()

    loader = ModelLoader(User)
    with timer():
        print("Partial loader")
        timeit.timeit(lambda: loader.do_load(row, ctx), number=20000)

Before:

Full model
CPU time: 1.01 s, Real time: 1.05 s
Single column loader
CPU time: 0.47 s, Real time: 0.49 s
Partial loader
CPU time: 0.74 s, Real time: 0.75 s

After:

Full model
CPU time: 0.60 s, Real time: 0.60 s
Single column loader
CPU time: 0.37 s, Real time: 0.37 s
Partial loader
CPU time: 0.49 s, Real time: 0.49 s

@wwwjfy
Copy link
Member

wwwjfy commented Jul 4, 2021

Looks good, thanks!

@wwwjfy wwwjfy merged commit 75fab19 into python-gino:master Jul 4, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants